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"Solid Tumor, Cortical Function, and Nerve Imaging" 

5 TECHNICA L FIELD OF THE INVENTION 

The present invention provides a method for real-time detection of solid tumor 
tissue, phis an ability to grade and characterize tumor tissue. The present invention 
further provides a method for real-time mapping of functional and dysfunctional 
cerebral cortex and nervous tissue. The present invention further provides a device for 
10 real-time detection and optical imaging for the inventive methods. 

BACKGRO UND OF TTTE INVENTION 

A primary goal of neurological surgery is the complete removal of abnormal or 
pathological tissue while sparing normal areas. Hence, the neurosurgeon attempts to 
identify boundaries of pathological or dysfunctional tissue and to map adjacent areas of 
the cortex committed to important functions, such as language, motor and sensory areas 
so that pathological/dysfunctional tissue is removed without removing functional areas. 

Incidence rates for primary intracranial brain tumors are in the range of 50-150 
cases per million population or about 18,000 cases per year (Berens et al. 1990). 
Approximately one half of brain tumors are malignant. The incidence of malignant brain 
tumors in adults is predominantly in the age range of 40-55 years while the incidence of 
more benign tumors peaks near 35 years of age. A primary means for treatment of such 
tumors is surgical removal. Many studies have shown that when more of the total 
amount of tumor tissue is removed, the better the clinical outcome. For gross total 
25 resections of tumors, the 5-year survival rate is doubled when compared to subtotal 

resection. Both duration of survival and independent status of the patient are prolonged 
when the extent of resection is maximized in malignant gliomas. Current intraoperative 
techniques do not provide rapid differentiation of tumor tissue from normal brain tissue, 
especially once the resection of the tumor begins. Development of techniques that 
enhance the ability to identify tumor tissue intraoperative^ may result in maximizing the 
degree of tumor resection and prolonging survival. 

Of the 500,000 patients projected to die of systemic cancer per year in the 
United States, approximately 25%, or over 125,000 can be expected to have 
intracranial metastasis. The primary focus for surgery in this group is in those patients 
with single lesions who do not have widespread or progressive cancer. This group 
represents about 20-25% of patients with metastases (30,000), however, the actual 
number of patients that are good candidates for surgery is slightly smaller. Of those 
patients undergoing surgery, one half will have local recurrence of their tumor at the 
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site of operation, while the other half -will develop aturaor elsewhere. The feet that 
about 50% of the surgeries foil at the site of operation means that an improved ability to 
remove as much tumor as possible by detecting and localizing tumor margins during 
tumor removal could potentially decrease the incidence of local recurrence. 
5 Thus, for both primary and metastatic tumors, the more tumor tissue removed, 

the better the outcome and the longer the survival. Further, by maximizing the extent of 
resection, the length of functional, good quality survival is also increased. 

Most current tumor imaging techniques are performed before surgery to provide 
information about tumor location. Presuigery imaging methods include magnetic 

10 resonance imaging (MRI) and computerized tomography (CT). In the operating room, 
only intraoperative ultrasound and stereotaxic systems can provide information about 
the location of tumors. Ultrasound shows location of the tumor from the surface but 
does not provide information to the surgeon once surgery begins to prevent destruction 
of important functional tissue while permitting maximal removal of tumor tissue. 

J 5 Stereotaxic systems coupled with advanced imaging techniques have (at select few 

hospitals) been able to localize tumor margins based upon the preoperative CT or MRI 
scans. However studies (Kelly, 1990) have shown that the actual tumor extends 2-3 cm 
beyond where the image enhanced putative tumor is located on preoperative images. 
Therefore, the only current reliable method to determine the location of tumors is by 

20 sending biopsies during surgery multiple histological margin sampling) and waiting 
for results of microscopic examination of frozen sections. Not only is k not advisable to 
continually take breaks during singery, but such biopsies are, at best, an estimation 
technique and are subject to sampling errors and incorrect readings as compared to 
permanent tissue sections that are available about one week later. Thus, a surgeon 

25 often relies upon an estimation technique as a guide when patient outcome is dependent 
upon aggressive removal of tumor tissue. Surgeons have difficult decisions between 
aggressively removing tissue and destroying surrounding fractional tissue and may not 
know the real outcome of their procedure until one week later and this may require an 
additional surgical procedure. 

30 Multiple histological margin sampling suffers several drawbacks. First this is a 

time-consuming procedure as it can add about 30 to 90 minutes (depending upon the 
number of samples taken) to a surgical procedure when the patient is undo- anesthesia. 
Second, this procedure is prone to errors as a pathologist must prepare and evaluate 
samples in short order. Third, it is certainly the case that margin sampling does not 

35 truly evaluate all regions surrounding a primary tumor as some areas of residual tumor 
can be missed due to sampling error. Fourth, increased time for margin sampling is 
expensive as operating room time costs are high and this leads to increased overall 
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medical costs. Moreover, increased operating room time for the patient increases the 
probability of infection. 

Other techniques developed to. improve visual imaging of solid tumor masses 
during surgery include determining the shape of visible luminescence spectra from 
5 normal and cancerous tissue. According to US Patent 4,930,516, in cancerous tissue 
there is a shift to blue with different luminescent intensity peaks as compared to normal 
tissue. This method involves exciting tissue with a beam of ultraviolet (UV) light and 
comparing visible native luminescence emitted from the tissue with a historical control 
from the same tissue type. Such a procedure is fraught with difficulties because a real 

10 time, spatial map of the tumor location is not provided for the use of a surgeon. 

Moreover, the use of UV light for an excitation wavelength can cause photodynamic 
ciian 8es to normal cells, is dangerous for use in an operating room, and penetrates only 
C superficially into tissue and requires quartz optical components instead of glass. 

Therefore, there is a need in the art for a more comprehensive and fester 

15 technique and a device for assisting such a technique to localize for solid tumor 

locations and map precise tumor margins in a real-time mode during surgery. Such a 
device and method should be further useful for inexpensive evaluation of any solid 
tumor (e.g., breast mammography) by a noninvasive procedure and capable of grading 
and characterizing the tumors. 

20 There is also a need to image brain functioning during neurosurgical procedures. 

For example, a type of neurosurgical procedure which also exemplifies these principles 
is the surgical treatment of intractable epilepsy (that is, epilepsy which cannot be 
controlled with medications). Presently, electroencephalography (EEG) and 
electrocorticography (ECoG) techniques are used prior to and during surgery for the 

25 purposes of identifying-areas of abnormal brain actrvity, such as epileptic foci. These 
measurements provide a direct measurement of the brain's electrical activity. 
( Intraoperative EEG techniques involve placing an array of electrodes upon the 

surface of the cortex. This is done in an attempt to localize abnormal cortical activity of 
epileptic seizure discharge. Although EEG techniques are of widespread use, hazards 

30 and limitations are associated with these techniques. The size of the electrode surface 
and the distance between electrodes in an EEG array are large with respect to the size 
of brain cells (e.g., neurons) with epileptic foci. Thus, current techniques provide poor 
spatial resolution (approximately 1 .0 cm) of the areas of abnormal cortical activity. 
Further, EEG techniques do not provide a map of normal cortical function in response 

35 to external stimuD (such as being able to identify a cortical area dedicated to speech, 
motor or sensory functions by recording electrical activity while the patient speaks). A 
modification of this technique, called cortical evoked potentials, can provide some 
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functional cortical mapping. However, the cortical evoked potential technique suffers 
from the same spatial resolution problems as the BEG technique. 

The most common method of intraoperative localization of cortical function in 
epilepsy and tumor surgery is direct electrical stimulation of the cortical surface with a 

5 stimulating electrode. Using this technique, the surgeon attempts to evoke either an 
observed motor response from specific parts of the body, or in the case of an awake 
patient, to generate specific sensations or cause an interruption in the patient's speech 
output Again, this technique suffers from the same problems as the EEG technique 
because ft offers only crude spatial localization of function. 

1 0 Possible consequences of the inaccuracies of all these techniques, when 

employed for identifying the portion of the cortex responsible for epileptic seizures in a 
patient, are either a greater than necessary amount of cortical tissue is removed possibly 
leaving the patient with a deficit in function, or that not enough tissue is removed 
leaving the patient uncured by the surgery. Despite these inadequacies, such techniques 

1 5 have been deemed acceptable treatment for intractable epilepsy. The same principles 
apply to tumor surgeries, however* intraoperative functional mapping is not performed 
routinely. 

In the past few years, researchers have been using imaging techniques in animal 
models to identify functional areas of cortex with high spatial resolution. One type of 

20 such technique uses a voltage-sensitive dye. A voltage-sensitive dye is one whose 
optical properties change during changes in electrical activity of neuronal cells. The 
spatial resolution achieved by these techniques is near the single cell level. Blasdel and 
Salama {Nature 321 ;579, 1986) used a voltage-sensitive dye (merocyanine oxazolone) 
to map cortical function in a monkey model. The use of these kinds of dyes would pose 

25 too great a risk for use in humans in view of their toxicity. Further, such dyes are 
bleached by fight and must be infused frequently. 

Recently, measurement of intrinsic signals have been shown to provide similar 
spatial resolution as voltage-sensitive dye imaging. Intrinsic signals are fight reflecting 
changes in cortical tissue partially caused by changes in neuronal activity. Unlike other 

30 techniques used for imaging neuronal activity, imaging intrinsic signals does not require 
using dyes (which are often too toxic for clinical use) or radioactive labels. For 
example, Grinvald et aL {Nature 324:361, 1986) measured intrinsic changes in optical 
properties of cortical tissue by reflection measurements of tissue in response to 
electrical or metabolic activity. Light of wavelength 500 to 700 nm may also be 

35 reflected differently between active and quiescent tissue due, to increased blood flow 
into regions of higher neuronal activity. Another aspect which may contribute to 
intrinsic signals is a change in the ratio of oxyhemoglobin to deoxyhemoglobin. 
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Ts'o ct al (Science 249:417, 1990) used a charge-coupled device (CCD) camera 
to detect intrinsic signals in a monkey model. However, this technique would not be 
practical in a clinical environment because imaging was achieved by implanting a 
stainless steel optical chamber in the skull and in order to achieve sufficient signal to 
5 noise ratios, Ts'o et al. had to average images over periods of time greater than 30 
minutes per image. By comparison to all other known techniques for localizing cortical 
function, imaging intrinsic signals is a relatively non-invasive technique. 

Mechanisms responsible for intrinsic signals are not well understood, possible 
sources of intrinsic signals include dilatation of small blood vessels, increased scattering 
10 of light from neuronal activity-dependent release of potassium, or from swelling of 
neurons and/or glial cells. 

Therefore, there is a need in the ait for a procedure and apparatus for real-time 
optical imaging of cortical tissue which can precisely and quickly distinguish normal and 
abnormal cortical tissue. There is also a need in the art for developing a method that 
15 can image intrinsic signals with high spatial resolution, provide immediate images and 
be compatible with normal procedures in the operating room This invention was made, 
in part, in an effort to satisfy this need. 

SUMMARY OF TBHE INVENTION 

20 The inventive method and device can be used to identify, grade and characterize 

solid tumors by imaging changes in electromagnetic absorption which reflects dynamics 
of dye perfusion through tissue, wherein the inventive device is able to differentiate 
tumor tissue from surrounding normal tissue with dynamic changes in optical signals 
during dye perfusion. Further, the inventive method and device can be used to identify 

25 areas of neuronal activity during neurosurgical procedures. In particular, this invention 
can be used by a neurosurgeon intraoperative^ to identify areas in the brain dedicated 
to important functions such as vision, movement, sensation, memory and language. 
Further the present inventive method and device can be used to detect areas of 
abnormal cortical activity, such as epileptic foci. Lastly, the present invention can be 

30 used to identify individual nerves during neurosurgical procedures for tumor removal or 
anastamoses of severed nerves. 

The present invention provides an apparatus for imaging tumor tissue or for 
real-time surgical imaging of cortical intrinsic signals or visualizing margins of solid 
tumor tissue from dynamic changes in optical signals during dye perfusion, comprising, 

35 a means for obtaining a series of analog video signals, and a means for processing the 
analog video signals into either an averaged control image or a subsequent averaged 
image, a means for acquiring and analyzing a plurality of subsequent images and 
averaged control images to provide a difference image, wherein the difference image is 
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processed to account for movement and noise and to amplify the changes across a 
dynamic range of the apparatus, and a means for displaying the difference image alone 
or superimposed over an analog video image. 

The present invention further provides a method for imaging tumor margins and 
5 dimensions of solid tumor tissue located in an area of interest, comprising illuminating 
the area of interest with spatially even, intensive and non fluctuating light containing a 
wavelength of electromagnetic radiation (emr) (e.g., light) absorbed by a dye; obtaining 
a video signal of the area of interest as a series of frames and processing the series of 
frames into an averaged control image, administering the dye by bohis injection into 

1 0 vasculature circulating to the area of interest, obtaining a subsequent series of frames of 
the area of interest over time and processing the subsequent series of frames into a 
subsequent averaged image, comparing each subsequent averaged image with the 
averaged control image to obtain a series of difference images, and comparing each 
difference image for initial evidence of changed optical signals within the area of interest 

1 5 which is the outline of solid tumor tissue, wherein tumor tissue is characterized by 
different kinetics of dye uptake compared to normal tissue and a temporally changed 
pattern of altered absorption of light as a result of increased vascularity of solid tumor 
tissue. Examples of appropriate dyes include indocyanine, fluorescein, 
hematoporphyrin, and fluoresdamine. A preferred dye is indocyanine green which has a 

20 broad absorption wavelength range and a peak absorption in the range of 730 nm to 
840 nm. 

The present invention further comprises a method in real-time for optically 
imaging functional areas of interest of the cortex in a patient comprising illuminating the 
area of interest with high intensity emr containing near-infrared wavelengths of emr, 

25 obtaining a series of frames of the area of interest and processing the series of frames 
into an averaged control image, administering a stimulus paradigm to the patient to 
stimulate an intrinsic signal, obtaining a series of subsequent frames of the area of 
interest over time and processing the subsequent series of frames into a subsequent 
averaged image, comparing each subsequent averaged image with the averaged control 

30 image to obtain a series of difference images, and comparing each difference image for 
initial evidence of an intrinsic signal within the area of interest, whereby an intrinsic 
signal is characterized by a change in emr reflectance properties manifest as a signal in 
the difference image. 

The present invention further includes a method for imaging damage to a 

35 peripheral or cranial nerves comprising: (a) illuminating an area of interest with high 
intensity emr, wherein the area of interest comprises the peripheral nerve of interest 
including the suspected ate of damage and a region adjacent of the suspected site of 
damage; (b) obtaining a series of frames of the area of interest and processing the series 
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of frames into an averaged control image; (c) stimulating the peripheral or cranial nerve 
at a ate adjacent of the suspected damaged site; (d) obtaining a series of subsequent 
frames at the time of stimulation and processing the series of subsequent frames into a 
subsequent averaged image; and (e) obtaining a difference image by subtracting the 
5 averaged control image from the subsequent averaged image to visualize the active 
region of the peripheral or cranial nerve, whereby nerve blockage is visualized as the 
point along the nerve where the intrinsic signal from the stimulated nerve abruptly ends, 
or is altered, attenuated or diminished in the difference image. 

The present invention further includes a method for imaging tumor tissue 
10 surrounding or adjacent to nerve tissue to aid in selective resection of tumor tissue 
without destroying nerve tissue, comprising: (a) ffluminating an area of interest with 
high intensity emr containing wavelength of emr absorbed by a dye; (b) obtaining a 
( sen** of frames of the area of interest and processing the series of frames into an 

averaged control image; (c) stimulating the nerve; (d) obtaining a series of subsequent 
15 nerve frames and processing the subsequent series of nerve frames into a subsequent 
nerve averaged image; (e) obtaining a nerve difference image by subtracting the nerve 
averaged control image from the nerve subsequent averaged image to visualize the 
active nerve; (f) administering a dye into an artery feeding the area of interest; (g) 
obtaining a series of tumor subsequent frames and processing the tumor subsequent 
20 series of frames into a tumor subsequent averaged image; and (h) obtaining a tumor 
difference image by subtracting the tumor averaged control image from the tumor 
subsequent averaged image to create a tumor difference image that is capable of 
visualizing the tumor. Further, the tumor difference image and the nerve difference 
image can be superimposed upon each other to simultaneously visualize the relative 
25 locations of tumor tissue and nervous tissue. 

The present invention further comprises a method for enhancing sensitivity and 
contrast of the images obtained from tumor tissue or intrinsic signal difference images, 
comprising: (a) fllutninating an area of interest with a plurality of wavelengths of emr, 
wherein there is at least a first wavelength of emr and a second wavelength of emr; (b) 
obtaining a sequence of frames corresponding to each wavelength of emr, wherein a 
first sequence of frames is from the first wavelength of emr, the second sequence of 
frames is from the second wavelength of emr and so on; (c) processing the first 
sequence of frames into a first averaged control image, the second sequence of frames 
into a second averaged control image and so on; (d) stimulating for intrinsic signals or . . 
administering a dye for tumor tissue imaging; (e) obtaining a first series of subsequent 
frames using the first wavelength of emr, a second series of subsequent frames using the 
second wavelength of emr, and so on, and processing the first, second and so on 
subsequent series of frames into the first, second and so on subsequent averaged 
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images, respectively; (f> obtaining a first difference image by subtracting the first 
averaged control image from the first subsequent averaged image and a second 
difference image by subtracting the second averaged control image from the second 
subsequent averaged image, and so on; and (g) obtaining an enhanced difference image 
5 by ratioing the first difference image to the second difference image. Preferably, the 
monochromatic emr sources to illuminate the area of interest are from laser sources. 
This technique is useful for obtaining three dimensional information of the area of 
interest. 

10 BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 illustrates a view of human cortex just anterior to face-motor cortex 
with one recording (r) and two stimulating electrodes (s), and three sites (#1, #2, #3) 
where average percent changes were determined. The scale bar equals 1 cm. Averages 
of 128 images (4/sec) were acquired at 30 Hz and stored (1/sec). After acquiring 3-6 

15 averaged control images (5 sec/image), a bipolar cortical stimulation evoked 
epileptiform afterdischarge activity. 

Figure 1 A is a view of a human cortex just anterior to face-motor cortex with 
one recording electrode (r) and two stimulating electrodes (s), and four sites (the boxed 
areas labeled 1, 2, 3, and 4) where the average percent changes of absorption over these 

20 areas were determined. The cortex was illuminated with emr >690 nm. The scale bar is 
1 cm. 

Figure IB are plots of the percent change of emr absorption per second in the 
spatial regions of boxes 1 and 3 (as labeled in Figure 1 A). For both regions, the peak 
change is during the fourth stimulation trial (at 8 mA) in which the greatest amount of 

25 stimulating currenthad biduced the most prolonged epileptiform afterdischarge activity. 
The changes within box 3 were greater and more prolonged than those of box 1. Box 3 
was overlying the area of the epileptic focus (the excitable area of tissue possibly 
responsible for the patient's epilepsy). 

Figure 1 C show plots of the percent change of emr absorption per second in the 

30 spatial regions of boxes 1 and 4 (as labeled in Figure 1 A). Box 1 overlays and area of 
cortical tissue between the two stimulating electrodes, and box 4 overlays a blood 
vessel. The changes within box 4 are much larger and in the opposite direction of box 
1, Also these changes are graded with the magnitude of stimulating current and 
afterdischarge activity. Since the changes in box 4 are most likely due to changes of the 

35 blood-flow rate within a blood vessel, this plot shows that the invention can 
simultaneously monitor cortical activity and blood-flow. 

Figure ID shows plots of the percent change of emr absorption per second in 
the spatial regions of boxes 1 and 2 (as labeled in Figure 1 A). Note that although these 
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two areas are nearby each other, their optical changes are in the opposite direction 
during the first three stimulation trials using 6 mA current. The negative going changes 
within the region of box 2 indicate that the invention may be used to monitor inhibition 
of cortical activity as well as excitation. 
5 Figure 2 illustrates a spatial map of stimulation induced epileptiform activity. 

This Figure shows a comparison between different degrees of activation for both spatial 
extent and amplitude of optical change graded with the extent of cortical activity. 
Specifically, Figure 2 shows percentage difference images from various times during 
two stimulation trials (definition of stimulation trial is given in description of Figure 1) 
10 from those described in Figure 1. The top 3 pictures (A2, B2, and C2) are from 
stimulation trial 2 where 6 mA cortical stimulation evoked a brief period of 
afterdischarge. These are compared to the bottom three pictures (A4, B4, and C4) 
C which are from stimulation trial 4 showing the optical changes evoked by cortical 

stimulation at 8 mA. Figures 2, A2 and A4 compare control images during rest. 
15 Figures 2, B2 and B4 compares the peak optical changes occurring during the 

epileptiform afterdischarge activity. Figures 2, C2 and C4 compares the degree of 
recovery 20 seconds after the peak optical changes were observed. The magnitude of 
optical change is indicated by the gray-scale bar in the center of the Figure. The arrow 
beside this gray-scale indicates the direction of increasing amplitude. Each image maps 
an area of cortex approximately 4 cm by 4 cm. 

Figure 3 shows a sequence of dynamic changes of optical signals identifying 
active areas and seizure foci. This Figure shows eight percentage difference images 
from the stimulation trial 2 described in the previous two Figures. Each image is 
integrated over a two second interval. The focal area of greatest optical change is in the 
center of images 3, 4, and 5, indicating the region of greatest cortical activity. This 
region is the epileptic focus. The magnitude of optical change is indicated by the gray- 
scale bar on the right side of the Figure. The arrow beside this gray-scale indicates the 
direction of increasing amplitude. Each image maps an area of cortex approximately 4 
cm by 4 cm. 

30 Figure 4 illustrates a real-time sequence of dynamic changes of stimulation- 

evoked optical changes in human cortex. Figure 4, panels 1 through 8, show eight 
consecutive percentage difference images, each image is an average of 8 frames (< 1/4 
second per image). The magnitude of optical change is indicated by the gray-scale bar 
in the center of the Figure. The arrow beside this gray-scale indicates the direction of 
35 increasing amplitude. Each image maps to an area of cortex that is approximately 4 cm 
by 4 cm. This Figure demonstrates that the inventive device and method can be used to 
map, in real time, dynamics of optical changes and present such information to the 
surgeon in an informative format. 
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Figure 5 shows an activation of somatosensory cortex by stimulation of a 
peripheral nerve in an anesthetized rat (afferent sensory input by directly stimulating the 
sciatic nerve in the hind limb of a rat). The leftmost image is a gray-scale image of hind 
limb somatosensory cortex in an anesthetized rat. The magnification is sufficiently high 
5 so that individual capillaries can be distinguished (the smallest vessels visible in this 
image). The center image is an image of a percentage difference control optical image 
during rest. The magnitude of optical change is indicated by the gray-scale bar in the 
center of this image. The arrow beside this gray-scale indicates the direction of 
increasing amplitude. The rightmost image is a percentage difference map of the optical 

1 0 changes in the hind limb somatosensory cortex during stimulation of the sciatic nerve. 
Figure 6 shows fonctional mapping of human language (Brocks area) and 
tongue and palate sensory areas in an awake human patient During three "tongue 
wiggling" trials images were averaged (32 frames, 1 sec) and stored every 2 seconds. A 
tongue wiggling trial consisted of acquiring 5-6 images during rest, then acquiring 

1 5 images during the 40 seconds that the patient was required to wiggle his tongue against 
the roof of his mouth, and then to continue acquiring images during a recovery period. 
The same patient engaged in a "language naming" trial. A language naming trial 
consisted of acquiring 5-8 images during rest (control images - the patient silently 
viewing a series of blank slides), then acquiring images during the period of time that 

20 the patient engaged in a naming paradigm (naming a series of objects presented with a 
slide projector every 2 seconds, selected to evoke a large response in Broca's area), and 
finally a series of images during a recovery period following a time when the patient 
ceased his naming task (again viewing blank slides while remaining silent). Images Al 
and Bl are gray-scale images of an area of human cortex with left being anterior, right- 

25 posterior, top-superior, and the Sylvan fissure on the bottom. The two asterisks on Al, 
Bl, A2, and B2 serve as reference points between these images. The scale bars in the 
lower right corner of Al and B I are equal to 1 cm. In Al, the numbered boxes 
represent sites where cortical stimulation with electrical stimulating electrodes evoked 
palate tingling (IX tongue tingling (2), speech arrest-Broca's areas (3,4) and no 

30 response (1 1, 12, 17, 5-premotor). Image A2 is a percentage difference control image 
of the cortex during rest in one of the tongue wiggling trials. The gray-scale bar on the 
right of A2 shows the relative magnitude of the color code associated with images A2, 
A3, B2 and B3. Image A3 is a percentage difference map of the peak optical changes 
occurring during one of the tongue wiggling trials. Areas identified as tongue and 

35 palate sensory areas by cortical stimulation showed a large positive change. 

Suppression of baseline noise in surrounding areas indicated that, during the tongue 
wiggling trials, language-motor areas showed a negative-going optical signaL Image 
B2 is percentage difference control image of the cortex during one of the language 
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naming trials. Image B3 is a percentage difference image of the peak optical change in 
the cortex during the language naming task. Large positive-going signals are present in 
Broca's area. Negative-going signals are present in tongue and palate sensory areas. 

Figure 7 shows a time course and magnitude plot of dynamic optical changes in 
5 human cortex evoked in tongue and palate sensory areas and in Broca's area (language). 
This Figure shows the plots of the percentage change in the optical absorption of the 
tissue within the boxed regions shown in Figure 6, images Al and Bl, during each of 
the three tongue wiggling trials and one of the language naming trials (see, description 
of Figure 6). Diagram 7A shows the plots during the three tongue wiggling trials 

10 averaged spatially within the boxes l t 2, 3, and 4 as identified in Figure 6, image Al. 
Diagram 7B shows the plots during one of the language naming trials averaged spatially 
within the boxes 1-7 and 1 7. 
C Figure 8 illustrates an optical map of a cortical area important for language 

comprehension (Wernicke 1 * area) in an awake human. Figure 8 image A shows the 

1 5 cortical surfece of a patient where the anatomical orientation is left-anterior, bottom- 
inferior,, with the Sylvan fissure running along the top. After optical imaging, all 
cortical tissue to the left of the thick line was sutgically removed. Sites #1 and #2 were 
identified as essential for speech (e.g., cortical stimulation blocked ability of subject to 
name objects). At site #3, one naming error in 3 stimulation trials was found. As the 

20 surgical removal reached the area labeled by the asterisks on the ttfck line, the patient's 
language deteriorated. All the unlabeled sites in Figure 8A had no errors while naming 
slides during cortical stimulation. Figure 8, image B shows an overlay of a percentage 
difference image over the gray-scale image of the cortex acquired during a language 
naming trial (see, Figure 6 for description of the language naming trial). The magnitude 
^25 of the optical change is shown by the gray-scale bar on the lower right of the image. 
This image demonstrates how a surgeon might use this invention intraoperatively to 
^ map language cortex. 

Figure 9 illustrates a timecourse and magnitude of dynamic optical changes in 
human cortex evoked in Wernicke's area (language comprehension). Figure 9A shows 

30 plots of percentage change b optical absorption of tissue within the boxed regions 
shown in Figure 8. The plots of boxes 1 and 2 overlay essential language sites, and 
boxes labeled 4, 5, and 6 overlay secondary language sites. Each of these five sights 
showed significant changes occurring while the patient was engaged in a language 
naming task. Figure 9B show percentage changes from the six unlabeled boxes shown 

35 in Figure 8, There were no significant increases or decreases within these anterior sites. 

Figure 10 illustrates differential dynamics of dye to identify low grade human 
CNS tumor This series of images are from a patient having a low grade CNS tumor 
(astrocytoma, grade 1). In Figure 10A (upper left) the lettered labels placed upon the 
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brain by the surgeon overlay the tumor as identified intraoperatively by ultrasound. 
However, tumors of this type and grade are notoriously difficult to distinguish from 
normal tissue once the surgical removal of the tumor has begun. Figure 10B (middle 
left) shows a difference image taken approximately 15 seconds after intravenous 

5 injection of dye (todocyanine green at 1 mg/kg). Figure IOC (lower left) shows the 
difference image about 30 seconds after dye administration. The area of the tumor 
tissue showed the first tissue staining. Figure 10D (top right) shows that in this low 
grade tumor, all tissue (both normal and abnormal) showed staining at 45 sec after dye 
administration. Figure 10E (middle right) is one minute after dye administration and 

10 Figure 10F is five minutes after dye administration (showing complete clearance in this 
low grade tumor). These data show that indocyanine green enters low grade tumor 
tissue fester than normal brain tissue, and may take longer to be cleared from benign 
tumor tissue than normal tissue, providing utility to image even low grade tumors, and 
to distinguish intraoperatively, low grade tumor tissue from surrounding norma! tissue. 

15 Figure 1 1 illustrates that differential dynamics of dye identify malignant human 

CNS tumor. The series of images in this Figure are from the cortex of a patient with a 
malignant CNS tumor (glioblastoma; astrocytoma, Grade IV). Figure 1 1 A (upper left) 
shows a gray-scale image in which malignant brain tumor tissue was densest in the 
center and to the right but elsewhere was mostly normal tissue (as was shown by 

20 pathology slides and flow cytometry available one week after surgery). Figure 11B 
* (middle left) is the difference image at 15 seconds after intravenous injection of 
indocyanine green, showing the dynamics of dye perfusion in the first seconds in 
malignant tissue are similar to those in the first few seconds of benign tumor tissue (see 
Figure 1 1C). Figure 1 1 C (lower left) shows that at 30 seconds the malignant tissue is 

25 even more intense by comparison to the normal tissue. Figure 1 ID (upper right, 1 

minute after dye injection) and 1 IE (lower right, 10 minutes after dye injection) show 
that unlike benign tumor tissue, in malignant tumor tissue, dye is retained significantly 
longer, and in some cases, continues to sequester in the malignant tumor tissue over 
longer periods of time. These data illustrate a utility to identify malignant tumor tissue, 

30 distinguish intraoperatively between normal and malignant tumor tissue, and to 

distinguish between the various grades of tumor (e.g., normal vs. benign vs. malignant). 

Figure 12 shows that differential dynamics of dye identify small remnants of 
tumor tissue in the margin of a resected malignant human CNS tumor. The images are 
from an area of interest where a tumor was surgically resected and biopsies were taken - . 

35 for multiple histological margin sampling. The area of interest was thought to be free of 
tumor tissue after the surgical removal of the tumor. Normally, in this size of a 
resection margin, only a single frozen sample would be taken for pathology analysis. 
For the purpose of this study, five biopsies were taken from the margin to aid in 
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correlating the histology with the map obtained by the invention. Figure 12A (top left) 
shows a gray-scale image of the tumor margin. Figure 12B shows the margin with 
labels that the surgeon placed directly on brain. The purpose of these labels were to 
identify where the surgeon was going to remove biopsy samples for histological analysis 
5 after difference images were acquired with the inventive device. Figure 12C (lower left) 
shows the difference image 1 minute after intravenous injection of dye and Figure 12D 
(lower right) shows the difference image 10 minutes after dye injection. These post-dye 
difference images reveal a number of sights that contain tumor tissue as well as areas of 
normal tissue. The accuracy of the optical imaging was confirmed post operatively by 

10 analysis of the biopsies. Note that a small area on the lower right of Figure 12D 
indicates a possible region of tumor tissue that would not have been biopsied by the 
surgeon. Hence, even in the case of extensive biopsy, the sampling error exceeds the 
accuracy of the invention. These data show a utility to identify small remnants of tumor 
tissue in a tumor margin after resection of a tumor. 

15 Figure 13 shows that differential dynamics of dye can identify and characterize 

tumors in human patients that do not contrast enhance with MRJ imaging. A 
proportion of non-benign tumors are not observable with present MRI imaging 
techniques. The images in this Figure are from a patient whose tumor did not contrast 
enhance with MRI. This lack of enhancement is usually typical of benign tumors. 

20 However, optical imaging was able to identify this tumor as a non-benign type (an 
anopiastic astrocytoma as shown one week later by pathology and flow cytometry). 
Figure 13 A shows the gray-scale image of the area of interest. Figure 13B shows the 
difference image prior to dye injection. Figure 13C shows the area of interest 1 minute 
after intravenous dye injection, and Figure 13D shows the area of interest 5 minutes 

25 after dye injection. *fote"that the dye is retained in this tissue for a significant time. As 
shown in figures 10, 1 1, and 12, this dynamic trait is a characteristic of a non-benign 
tumor. 

Figure 14 shows non-invasive imaging of dye dynamics and identification of 
glioma through the intact cranium. This figure demonstrates that the invention can be 
30 used to identify tumors through the intact cranium. Figure 14A is a gray-scale image of 
the cranial surface of a rat. The sagital suture runs down the center of the image. 
Tumor cells had been injected into the left side some days earlier so that this animal had 
developed a ghoma.on the left hemisphere of its brain. The right hemisphere was 
normal. Box 1 lays over the suspect region of brain tumor, and box 2 lays over normal 
tissue. Figure 14B is a difference image 1 second after indocyanine green dye had been 
intravenously injected into the animal The region containing tumor tissue became 
immediately visible through the intact cranium. Figure 14C shows that 5 seconds after 
dye injection the dye can be seen to profuse through both normal and tumor tissue. 
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Figure I4D shows that 1 minute after dye injection, the normal tissue had cleared the 
dye, but dye was still retained in the tumor region. The concentration of dye in the 
center of this difference image was dye circulating in the sagital sinus. 

Figure 15 illustrates dynamic information of dye uptake and clearance in tumor 
5 vs. non-tumor tissue through the intact skull. This is a plot of an average of the 

percentage change in emr absorption average over the spatial areas indicated by boxes 1 
and 2 from Figure 14A. The increase in absorption was a function of the concentration 
of dye in the tissue at a particular time. The graph labeled "extracranial tumor" is a plot 
of the dynamics of the absorption changes within box 1 from Figure 14A, The graph 

10 labeled "extracranial: normal" is a plot of the dynamics of the absorption change within 
box 2 from Figure 14A. 

Figure 16 shows a spatial map of dynamic changes in tumor vs. non-tumor areas 
in the rat glioma model. The sequence of images in this figure demonstrate the dynamic 
differences of the absorption changes due to dye between tumor and non-tumor tissue. 

15 Figure 16A shows a gray-scale image of the area of interest. This is the same animal as 
shown in Figure 14, however the cranium has now been removed so as to expose the 
left hemisphere containing the glioma, and the right hemisphere containing normal 
tissue. Box 1 overlays the tumor, Box 2 the tumor-surround, and Box 3 overlays 
normal tissue. Figure 16B shows the difference image of the area of interested 1 second 

20 after 1 mg/kg of indocyanine green had been intravenously injected into the animal. 
During this initial time, the tumor tissue was the first to show a measurable optical 
change indicating the uptake of dye occurs first in the tumor tissue. The gray-scale bar 
indicated the relative magnitude of the optical changes in the sequence of difference 
images. Figures 16C and 16D show difference images of the area of interest 4 seconds 

25 and 30 seconds, respectively, after dye injection. At these intermediate stages dye 
appears to collect in both normal and tumor tissue. Figures 16E and 16F show 
difference images of the area of interest 1 minute and 5 minutes, respectively, after 
injection of dye. At these later times, it becomes clear that dye was still collecting in 
tumor tissue even thought it was being cleared from normal tissue. 

30 Figure 1 7 shows dynamic information of dye uptake and clearance in tumor vs. 

non-tumor tissue. This is a plot of an average of the percentage change in emr 
absorption averaged over the spatial areas indicated by boxes 1, 2, and 3 from Figure 
16A The increase in absorption was a function of the concentration of dye in the tissue 
at a particular time. The graph labeled "tumor tissue" is a plot of the dynamics of the 

35 absorption changes within box 1 from Figure 16A. The graph labeled "tumor surround" 
is a plot of the dynamics of the absorption changes within box 2 from Figure 16A The 
graph labeled "normal brain" is a plot of the dynamics of the absorption changes within 
box 3 from 16A 
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Figure 18 shows dynamic imaging ofdye uptake reveals residual traces of tumor 
cells in resetted tumor margins. This is a continuation of the study on the same animal 
shown in Figures 14 through 17. Figure 18A shows a higher magnification image of the 
left hemisphere tumor margin of the animal after the tumor has been resected. Boxes 1 
5 are over areas that contain small traces of residual tumor cells, and boxes 2 are over 
areas containing only normal tissue. The gray-scale bar indicates the magnitude of 
optical change in the difference images. Figures 18B, 18C, and 18D show difference 
images of the tumor margin 4, 30, and 60 seconds after intravenous dye injection 
respectively. Minute biopsies were taken from areas that showed preferred dye 

10 containment and from areas from which the dye cleared rapidly. These biopsies were 
analyzed blindly and later correlated to the location from which the biopsies were taken. 
Those biopsies taken from areas which cleared dye were shown to contain only normal 
cells, whereas biopsies taken from areas which sequestered dye were shown to contain 
tumor cells. Extremely small islands of residual tumor, can be mapped within the tumor 

IS margins. 

Figure 19 shows dynamic information of dye uptake and clearance in tumor vs. 
non-tumor tissue. This is a plot of an average of the percentage change in emr 
absorption average over the spatial areas indicated by boxes 1 and 2 from Figure 1 8 A 
The increase in absorption is a function of the concentration of dye in the tissue at a 

20 particular time. The graph labeled "margins tumor • is a plot of the dynamics of the 

absorption changes within box 1 from Figure 18A The graph labeled "margins normal- 
is a plot of the dynamics of the absorption changes within box 2 from Figure 1 8A. This 
data as well as that from Figure 18 show that the inventive device and method are able 
to distinguish tumor from non-tumor tissue within tumor margins with extremely high 

25 spatial and temporal resolution. 

DETATLED DESCRIPTION OF TTTF, INVENTION 

The present invention provides an apparatus for imaging neuronal intrinsic 
signals in real time and for determining the presence, size, margins, dimensions, and 

30 grade of a solid tumor mass using a dye. The present invention further provides a 

method for functional mapping of the cortex in a patient by mapping intrinsic signals in 
real time, a method for determining the presence, size, location, and grade of solid 
tumor tissue in real time without the sampling errors of biopsies or the delay of and 
possible misdiagnosis of the pathologist's frozen section analysis, and a method for 

35 imaging nerve tissue that may be physically damaged or surrounded by and adjacent to 
tumor cells. The inventive methods employ a similar apparatus, comprising a series of 
components, including video input hardware and dedicated image processing hardware. 
The video input hardware is, for example, a photo-detector, such as a CCD (charge 
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coupled device) camera (preferably a COHU 6510 CCD Monochrome Camera with a 
COHU 6500 electronic control box made by COHU Electronics San Diego, CA). In 
some cameras the analog signal is digitized 8-bits deep on an ADI board (analog-to- 
digital board). The dedicated image processing hardware is generally controlled by a 

5 "host computer 11 . The host computer is any common general computer (such as an 
IBM PC type with an Intel 386, 486 or better microprocessor or Sun SPARC) that is 
interfaced with the dedicated imaging hardware and sends commands to the imaging 
hardware that direct data flow, computations, image acquisition and the like. Thus, the 
host computer directs the actions of the imaging hardware and provides the user 

10 interface. 

Definitions 

The following are definitions of commonly used terms and that are applied in 
this application according to their art-accepted usage, such as described in Inoue, Video 
1 5 Microscopy Plenum Press, New York, 1 989. 

Area of Interest is that area of tissue that comprises the subject of the image. 
Arithmetic Lope Unit (ALU) is the hardware component that performs a variety 
of mathematical and logic operations {e.g., sum, difference, exclusive or, multiply by a 
constant, etc.) on the image signal at extremely high speeds. 
20 Averaged Control Image is that updateable image that is the average of a series 

of real time images over a period of time. 

Charge Co upled Device (CCD) is a photo-sensitive silicon chip used in place of 
a pickup tube in miniature video cameras. 

Difference Image is the manipulated image created by adding or subtracting a 
25 subsequent image or a particular image in time from an averaged control image. 
Frame is a single digitized array of single video pictures. 
Frame Buffer is a piece of hardware that serves as a temporary storage of a 
frame, such as an averaged control image, a subsequent image or a difference image. 
Geometric Transformation (Gonzalez and Wintz, Digital Image Processing, 
30 Addison-Wesiey Publishing Co., Reading, 1987) generally modify spatial relationships 
between pixels in an image. For this reason, geometric transformations are often called 
"rubber sheet transformations" because they can be viewed as the process of "printing" 
an image on a sheet of rubber and stretching this sheet according to a predefined set of 
rules. As applied to video imaging, subsequent images can be viewed as having been 
35 distorted due to movement and it is desirable to "warp* 1 these images so that they are 
similar to the control images. Geometric transformations are distinguished from "point 
transformations" in that point transformations modify a pixel's value in an image based 
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* 

solely upon that pixel's value and/or location and no other pixel values are involved in 
the transformation. 

Image is a frame or composition of frames that have been altered after 
digitization, such as processing a sequence of frames into an averaged control image or 
5 a subsequent averaged image. 

fotPflgfc Signal means a detectable change in reflectance properties of neuronal 
tissue due to endogenous physiologic activity. Possible causes of intrinsic signals 
include, for example, membrane depolarization, glial cell swelling, ion flux across 
neuronal membranes, blood volume changes, blood deoxygenation (hemoglobin to 
10 deoxyhemoglobin), tissue oxygenation and combinations thereof. 

Linear Histogram Stretch is a transformation m iMrk *U+ y^] v ^ fr^ veen fry? 
points (high, low) are remapped to cover a full range of values (i.e., dynamic range). 
For example, the low value is mapped to zero, the high to 255, and the intermediate 
values are mapped to linearly increasing brightness values. All brightness values below 
1 5 the low value are set to zero and all brightness values above the high value are set to the 
high value. 

jLook Up Table (LUT) is a piece of hardware that functions to store memory 
that directs conversion of the gray value of each pixel into another gray value or color 
that is specified by the LUT. The LUT can be programmed to manipulate image 
20 contrast, threshold an image, apply pseudocolor and the like (such as a convenient 
implementation method for point processing algorithms). In the case of the present 
invention, the LUTs are, preferably, implemented for speed on an ADI and/or ALU 
boards. 

Paradigms cause a change in electrical activity of an area of cortical tissue 
25 dedicated to a specific function (e.g., speech, language, vision, etc) thus causing an 
increase- or decrease in what is called an intrinsic signal. 

Bxgl is the individual units of image in each frame of the digitized signal The 
intensity of each pixel is linearly proportional to the intensity of illumination before 
signal manipulation and corresponds to the amount of emr (photons) being reflected 
30 from a particular area of tissue corresponding to a particular pixel. It should be noted 
that an image pixel is the smallest unit of a digital image and hs output intensity can be 
any value. A CCD pixel is the smallest detecting element on a CCD chip and its analog 
output is linearly proportional to the number of photons it has detected. 

Processed Difference Tmage is the raw difference image that has been processed- 
35 or manipulated to filter out noise or movement and increase the dynamics of effect of 
different pixel values to illustrate events in the area of interest 

TjiTnor Margin is the area where the surgeon has resected the tumor. 
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Apparatus 

The inventive apparatus is made as one unit or a group of components. The 
first component is a high intensity emr. source. The emr source is for illuininating the 
cortical surface or area of interest, such as an area suspected of having solid tumor 
5 tissue. Different intrinsic signals can be illuminated by different wavelengths of emr. 
Moreover, the emr source must include the wavelengths of emr absorbed by the dye for 
the tumor imaging method For example, when the dye is indocyanine green, preferred 
wavelengths are from about 730 nm to about 840 nm. For other dyes, the preferred 
wavelengths of illuminating emr should include wavelengths at which the dye absorbs. 
1 0 The term emr instead of light is used because it is also possible to image in the infrared 
region of the spectrum outside of the visible light range. 

When determining intrinsic signals from the cortex, reflected emr can be filtered 
to allow for video imaging of only selected wavelengths of emr. Preferred selected 
wavelengths of emr include, for example, from about 500 nm to about 900 nm, or most 
15 preferably, the near infrared spectrum. Generally, longer wavelengths {e.g* 9 
approximately 800 nm) measure deeper cortical activity. 

Moreover, that part of the infrared spectrum in an invisible range of between 
0.75 to about 1000 micrometers allows for a determination of intrinsic signals through 
dura and skull, thereby allowing for a determination of intrinsic signals through intact 
20 skull and dura and without the risks associated with neurosurgery. When using this 
range of far infrared wavelengths, an IR detector is a different device than a CCD chip 
for a visible analog camera. IR detectors are made from materials such as indium 
arsenide, germanium and mercury cadmium telluride rather than silicon. IR detectors 
must be cryogenically cooled in order that they be sensitive to small changes in 
25 temperature. For example, one IR imaging system is an IRC-64 infrared camera 
(Cincinnati Electronics, Mason OH). 

As heat reaches the surface of the cortex, it emits electromagnetic radiation in 
the range of about 3-5 or 8-14 microns. Others have attempted to image this emitted 
radiation (see, for example, Gorbach et al., "Infrared Mapping of the Cerebral Cortex 1 ' 
30 Thermography 3 : 1 08, 1 989). However, according to the present invention these 
emitted wavelengths are filtered out and an IR detector instead of a CCD detector is 
used. An IR emr source is, for example, a Tunable IR Diode Laser from Laser 
Photonics, Orlando, FL. The imaged wavelengths are different from body heat and 
images of changes in absorption and emr scattering can be obtained according to the 
35 inventive method. In the case of tumor images through intact skin and possibly bone, a 
dye that absorbs m the IR can be used (e*g. y indocyanine green). Other useful dyes 
include, for example, Photofrin® derived from a hematoporphyrin derivative (HPD) and 
absorbs light at 630 nm, mono espatyl chlorin-36 (NPeg, Nippon Petrochemical, Japan), 
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benzoporphyrin derivative (BPD, Quadra Logic Vancouver BC), Evans Blue, and 
combinations thereof. 

Preferably, the emr source is a high intensity, broad spectrum emr source, such 
as a tungsten-halogen lamp and a cutoff filter for all wavelengths below 695 nm. Most 
5 preferably, the emr source is directed to the area of interest by a fiber optic means. An 
example of such a emr source is a fiber optic emr passing through a beam splitter, 
controlled by a D.C. regulated power supply (Lambda, Inc.) and passed through a 695 
nm longpass filter. 

The inventive apparatus includes a means for obtaining an analog video signal of 

JO the cortex or area of interest. A preferred device for obtaining an analog video signal is 
a charge coupled device (CCD) video camera which creates an output video signal at 
30 Hz having, for example, 512 horizontal lines per frame using standard RS 1 70 
convention. One such device is a CCD-72 Solid State Camera (Dage-MTI Inc., 
Michigan City IN) and another such device is a COHU 6500 (COHU, San Diego CA). 

1 5 The area of interest must be evenly illuminated to better adjust the signal over a 

full dynamic range. If there is uneven illumination in the area of interest, it will limit the 
dynamic range. Preferably a high intensity and diffuse or even lighting system is used. 
Techniques to obtain even illumination over the area of interest include, for example, 
diffuse lighting, image processing algorithms to compensate for uneven illumination on 

20 a digitized image, a constant shade gray image marker point in the area of interest as a 
control point, a wavelength cutoff filter in front of the camera and/or emr source, or 
combinations thereof. Preferably, a regulated power supply will prevent fluctuations in 
exnr sources. A footplate system is an optical glass (sterile) contacting and covering the 
area of interest to provide a flatter contour. The footplate also retards tissue 

25 movement. 

The analog signal must first be adjusted to maximize sensitivity of detection (at 
the level of the analog signal and before digitizing) to amplify the signal and spread the 
signal across the full possible dynamic range, thereby increasing sensitivity of the 
apparatus. 60 Hz noise (such as from A.C. power lines) is filtered out in the camera 

30 control box by an analog filter. Such adjustments further serve to enhance, amplify and 
condition the analog signal from the CCD. One means for properly adjusting the input 
analog signal is to digitize this signal at video speed (30 Hz), and view the area of 
interest as a digitized image that is converted back to analog. 

It is important to compensate for small movements of tissue or the patient 

35 during the imaging process. Larger patient movements require a new orientation of the 
camera and obtaining a new averaged control image. Compensating for movement can 
be done by mechanical or computational means or both. Mechanical means include, for 
example, placing a footplate over the area of interest wherein the footplate comprises 
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sterilized optical quality glass in a framing device, and/or securing the camera and 
possibly the emr source to the skeletal frame of the patient, and combinations of both. 
When the camera and/or emr source are attached to the skeletal structure of the patient, 
any patient movements will not effect the image because the camera and illumination 
5 source will remain in a constant orientation to the area of interest The advantage of the 
footplate is that it retards tissue movement caused by arterial pressure and/or respiration 
and prevents changes due to evaporation of cerebrospinal fluid Computational means 
include, for example, using functional control points in the area of interest and 
triangulation-type algorithms to compensate for movements of these control or tie 

10 points, and "image warping" techniques whereby each subsequent image is registered 
geometrically to the averaged control image to compensate for movement, and 
combinations of both techniques. Hie image warping technique is described in, for 
example, Wolberg, "Digital Image Warping" IEEE Computer Society Press, Los ( 
Alimitos, C A 1990. The image warping technique can further indicate when movement 

15 has become too great for the averaged control image and that a new averaged control 
image must be taken. Control points can be placed directly in the area of interest, such 
as directly on the cortical surface for intrinsic signal analysis. For example, Goshtasby 
("Piecewise Linear ^Mapping Functions for Image Registration* in Pattern Recognition 
vol 19 pp 459-66, 1986) describes a method whereby an image is divided into 

20 triangular regions using control points. A separate geometrical transformation is 
applied to each triangular region to spatially register each control point to a 
corresponding triangular region in a control image. 

If the two images (averaged control image and subsequent image) are 
misaligned prior to subtraction, artifacts win result since the difference image will be 

25 more like a gradient image amplifying noise and edge information. Image misalignment 
can arise from patient motion, heartbeat and respiration. One solution is to fix the 
camera to a rigid assembly connected to the patient, such as his or her head such that ( 
any patient motion also moves the camera's field of view accordingly. Another solution 
is to perform real time motion compensation with motion detection and geometric 

30 transformation with the image processing board. Simple translation or more 

complicated (thus more accurate) unwarping can be implemented depending upon the 
input frame rate and amount of averaging. 

In the case of imaging tissue (either for neuronal activity or for dynamical 
imaging of dye flow through tissue) in a human subject, it is necessaty to compensate 

35 for the motion of the subject which may occur between the acquisition of consecutive 
images. 

For many types of images, it is possible to compensate by a geometrical compensation 
which transforms the image by translation in the x-y plane. In order for an algorithm 
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such as this to be feasible, it must be computationally efficient (preferably 
implementable in integer arithmetic), memory efficient, and robust with respect to 
changes in ambient light. 

One possible method would be to translate an image by 0 through k number of 
5 pixels in every possible direction with respect to the control image. For each of the 
(2*k + l)*(2k + 1) translations, make a subtraction image and calculate some metric to 
estimate the closeness to the control image. An ©cample of such a metric would be the 
variance of the subtraction image. The drawback of this method is that it is not efficient 
since for each of (2*k+l)*(2k+l) subtraction images, we would need to calculate the 

JO variance over 512*512 pixels. 

An efficient improvement of tins algorithm is to estimate the variance of the 
subtraction images by randomly selecting some small number of areas of interest (for 
example, 9 areas of interest), each area consisting of a small number of pixels (say 8 x 
8) from the image that one wishes to translate with respect to the control image. Also, 

15 choose some search depth ( for example, 10 pixels) over which to translate these small 
areas of interest with respect to their corresponding areas of interest in the control 
image. After translation in all possible directions for 0 through 10 pixels, choose the 
translation which minimizes the variance over the selected areas of interest. Since all 
the areas of interest are the same size, division is not necessary in the calculation of the 

20 variance which is to be ordered so that the minimal variance can be selected. Hence, all 
calculations can be carried out in integer arithmetic. Since the areas of interest are 
sufficiently small, most of the data can be read into the host computer's RAM limiting 
10 to the frame buffers and increasing speed. 

Another problem is guaranteeing uniformity m the illumination of the tissue 
^25 surface. Nonuniformity comes from fluctuation in the illumination source and intensity 
variations resulting from the three-dimensional nature of the tissue surface. Fluctuation 
in the Ulumination source is addressed by using a light feedback mechanism to regulate 
the power supply of the illumination source. Both of these problems can also be 
compensated for in the image processing module, 

30 The analog video signal is continuously fed into a means for processing the 

signal. One such means for acquiring and analyzing data is an image analyzer (e.g. 9 
Series 1 5 1 Image Processor, Imaging Technologies, Inc. Woburn MA). An image 
analyzer can receive and digitize an analog video signal with an analog to digital 
interface and perform such a function at a frame speed of about 1/3 0th of a second 
35 30 Hz or "video speed"). Processing the signal involves first digitizing the signal 

into a series of pixels or small squares assigned a value On a binary system) dependent 
upon the number of photons {i.e., quantity of emr) being reflected off tissue from the 
part of the area of interest assigned to that pixel. For example, in a standard 512 x 512 
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image from a current technology CCD, there would be 262, 144 pixels per image. In an 
8 bit system, each pixel is represented by 8 bits. One can cool the CCD to reduce 
thermal noise. 

Preferably, the signal processing means includes a programmable look-up table 

5 (e.g., CM150-LUT16, Imaging Technology, Woburn, MA) initialized with values for 
converting gray coded pixel values, representative of a black and white image, to color 
coded values based upon the intensity of each gray coded value. This provides image 
enhancement via an image stretch. An image stretch is a technique whereby the highest 
and lowest pixel intensity values used to represent each of the pixels in a digital image 

10 frame are determined over a region of the image frame which is to be stretched, 

Stretching a selected region over a larger range of values permits, for example, easier 
identification and removal of relatively high, spurious values due to noise (e>g* 9 glare). 

Each image received is stored in the frame buffer, preferably within the context ( 
of a CPU as a frame of data elements represented, for example, as a 512 by 512 array of 

1 5 pixels. Each pixel has a 8 bit value corresponding to one of 256 levels of gray. 

The processing means further includes a plurality of frame buffers having frame 
storage arras for storing frames of digitized image data received from the A/D interface. 
The frame storage area comprises at least one megabyte of memory space, and 
preferably at least 8 megabytes of storage space. An additional 16-bit frame storage 

20 area is preferred as an accumulator for storing processed image frames having pixel 
intensities represented by more than 8-bits. The frame buffers are temporary fast 
memory. The processing means should include at least three frame buffers. One is for 
storing the averaged control image, another is for storing the subsequent image and a 
third is for storing a difference image between the averaged control image and the 

25 subsequent image. 

The processing means further includes an arithmetic logic unit (ALU) (e.g., 
ALU- 150 Pipeline Processor) for performing arithmetical (add, subtract, etc.) and 
logical (and, or, etc.) functions from data located in one or more frame buffers. An 
ALU is a fast processor. The ALU allows for image averaging in real time. For 

30 example, a newly incoming digitized image can be sent directly to the ALU and is added 
or subtracted to an averaged control image sitting in a frame buffer by passing both 
images through an ALU and adding them. After a last image is added, this 16 bit result 
can be sent again through an ALU which will divide this result by a constant (/.e., the 
total number of images). The output from the ALU is either stored in a frame buffer, - . 

35 sent for more processing, or used as hs own input and again combined with another 
image. 
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It is important to compensate for patient movement in the digitized images 
before subtracting such images. Thus, geometric transformations are appHed to the 
images so that they are geometrically registered prior to subtraction. 

The inventive apparatus can enhance processing speed to create a difference 
5 frame by adding a real time modular processor or fester CPU chip to the image 
processor. For example, one real time modular processor is a ISO RTMP-1 50 Real 
Time Modular Processor (Imaging Technology, Woburn, MA). 

The processing means further may include a means for performing a histogram 
stretch of the difference frames {e.g., Histogram/Feature Extractor HF 151-I-V 

1 0 module, Imaging Technology, Woburn MA) to enhance each difference image across its 
dynamic range. A linear histogram stretch is described in, for example, Green, Digital 
Image Processing: A Systems Approach, Van Nostrand Reinhold, New York, 1983. A 
histogram stretch assigns the brightest pixel, or one with the highest value in the 
difference image and assigns this the maximum value. The smallest pixel value is 

15 assigned the minimum value and every other value in between is assigned a linear value 
(for a linear histogram stretch or a logarithmic value for a log histogram stretch, etc.) in 
between the maximum and minimum values. This allows the difference image to fully 
utilize the full dynamic range which provide for absolute changes. 

The image processing system can use a variety of hardware that is available or 

20 under development. For example, the Texas Instrument Multimedia Video Processor 
(MVP) is under development for motion video applications. The MVP uses a highly 
parallel internal architecture, large on-chip memory, and extremely high bandwidth 
communication within CPU and between the CPU memory and I/O devices in order to 
provide in excess of 2 billion RISC-type operations per second perfbnnance necessary 

25 to support the requirement of real-time video compression standards and real-lime 
image capture, processing and visualization. For example, the hardware can comprise 
of printed circuit board modules with interfaces to a VME bus. A single chassis can 
house all of the modules and reside on a rack that is easily transportable in an operating 
room or between operating rooms, along with display monitors and peripheral input and 

30 output devices. The real time system, for example, comprises four boards for 
acquisition image processing, peripheral control and host computer. A minimal 
configuration with reducing processing capabilities comprises just the acquisition and 
host computer boards. The acquisition board comprises circuitry to perform real-time 
averaging of incoming video frames and allow readout of averaged frames at a 

35 maximum rate bus. A VME bus is preferred because of its high peak bandwidth 

(greater than 80 Mbytes/sec for the latest revision, VME64) and compatibility with a 
multitude of existing VME products. The acquisition board must also support many 
different types of cameras via a variable scan interface. A daughter board can support 
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the interfacing needs of many different types of cameras and supply variable scan signals 
to the acquisition motherboard. Preferably, the unit comprises a daughter board 
interfacing to an RS-1 70 A video signal to support a wide base of cameras. Other 
camera types, such as slow scan cameras with a higher spatial/contrast resolution andA>r 
5 better signal to noise ratio) can be developed and incorporated into the inventive device, 
as well as improved daughter boards to accommodate such improved cameras. 

The host computer comprises a single-board embedded computer with a VME 
interface. Preferably the host computer comprises a VME 64 interface, or a standard 
(IEEE 1014-1987) VME interface, depending upon bus bandwidth considerations. 

10 Example of host computer boards include, for example, Force SPARC/CPU-2E and 
HP9000 Model 7471. The user interface can be, for example, a Unix/X-Widow 
environment. The image processing board can be, for example, based upon Texas 
Instruments* MVP and other chips to perform real time image averaging, registration 
and other processing necessary to produce high quality difference images for 

15 intraoperative viewing. This board will also drive a 120 x 1024 RGB display to show a 
sequence of difference images over time with pseudo-color mapping to highlight tumor 
tissue. Preferably, a second monitor is used for the host computer to increase the 
ova-all screen real estate and smooth the user interface. The processing board (fully 
programmable) can support a VME64 master interface to control data transactions with 

20 the other boards. Lastly, a peripheral control board can provide electrical interfaces to 
control mechanical interfaces from the host computer. Such mechanical interfaces can 
include, for example, a computer-controlled, motor-driven syringe for dye injection, 
light source, and camera control box. 

The difference image signal is, preferably, further processed to smooth out the 

25 image and remove high frequency noise. For example, a lowpass spatial filter can block 
high spatial frequencies and/or low spatial frequencies to remove high frequency noises 
at either end of the dynamic range. This provides a smoothed-out processed difference 
image Cm digital format). The digitally processed difference image can be color-coded 
by assigning a spectrum of colors to differing shades of gray. This image is then 

30 converted back to an analog image (by an ADI board) and displayed for a real time 
visualization of differences between an averaged control image and subsequent images. 
Moreover, the processed difference image can be superimposed over the analog image 
to display regions upon a video display of the area of interest, those specific tissue sites 
where the dye may have a faster uptake or where an intrinsic signal may be occurring. 

35 The present invention further includes a means for subtractive processing of 

difference images to identify cortical areas of neuronal inhibition. Normally areas of 
increased neuronal activity result in an increase of the emr absorption capacity of 
neuronal tissue the tissue gets darker if visible light is used for emr illumination, or 
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an intrinsic signal increases in a positive direction). Similarly, a decrease in neuronal 
activity results in a decrease of emr absorption capacity of the tissue (i.e., the tissue 
appears brighter, or intrinsic signals become negative). For example, image A is a 
subsequent averaged image and image Bis an averaged control image. Normally, when 
5 a pixel in image A is subtracted from a pixel in image B and a negative value results, 
this value is treated as zero. Hence, difference images cannot account for areas of 
inhibition. However, the present invention provides a method for identifying both 
negative and positive intrinsic signals, by the method comprising: (a) subtracting image 
A (a subsequent averaged image) from image B (an averaged control image) to create a 

10 first difference image, whereby all negative pixel values are zero; and (b) subtracting 
image B from image A to create a second difference image whereby all negative pixel 
values are zero; and adding the first and second difference images to create a "sum 
difference image". The sum difference image shows areas of increased activity (i.e., 
color coded with wanner colors such as yellow, orange, red) and show areas of less 

1 5 activity or inhibition (Le. , color coded with colder colors such as green, blue, purple). 
Alternatively, one can overlay the first difference image on the second difference image. 
Either method provides an image of increased neuronal activity and decreased neuronal 
activity. 

Preferably, the processing means further includes an optical disk for storing 

20 digital image data, a printer for providing a hard copy of the digital and/or analog video 
image and a monitor to provide for the physician to continuously monitor the difference 
frame output (converted back to an analog signal) of the apparatus. The difference 
frame output may be superimposed upon the real time analog video image to provide a 
video image of the area of interest (e.g., cortical surface or suspected tumor site) 

25 superimposed with a color-coded difference frame, in frozen time, to indicate where 
regions of faster dye uptake have occurred and where there are intrinsic signals in 
response to some stimulus or paradigm. 

During a surgical procedure, there is often patient movement. In the case of an 
anesthetized patient, motion is often due to respiration and blood flow. In an awake 

30 patient, there will be additional movement. Movement must be compensated for in the 
digitized images so that the images are geometrically registered prior to subtraction. 
Geometric compensation is achieved by applying geometric transformations to the 
digitized images. One piece of image-processing hardware which can accomplish 
geometric transformations in real-time is a GP-150 Geometrical Processor board 

35 (Informatique et Techniques Avancees, Issy-les-Moulineaux, France). The GP- 1 50 
Processor board is compatible with Itex hardware and performs real time rotations, 
translations, zooms, and second degree distortion corrections at video rates with 
bilinear interpolation on 512 x 512 x 8-bit images. 
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Imagjns Methods 

The method for imaging a solid tumor involves periodically administering a dye 
by bolus injection into vasculature (e.g., artery or vein) perfusing the suspected tumor 

5 site in the area of interest Preferably* the dye has a relatively short half life (e.g., less 
than five minutes) and is rapidly cleared to allow for repeated administration. The video 
CCD of the inventive apparatus is focused upon the suspected solid tumor she (area of 
interest) and high intensity emr containing the wavelength absorbed by the dye 
illuminates the site. Just prior to administration of the dye, the first averaged image is 

]0 taken, digitized and stored in a frame buffer. The dye is injected quickly and rapidly as 
a bolus. Subsequent image frames are taken and stored and subtractively compared to 
produce difference images (e>g. 9 one or two per second) using the inventive processing 
means. Initial visualization of the dye will appear in the difference image first in tumor 
tissue because the dye perfuses more rapidly into tumor tissue. Solid tumor margins 

15 will be the first images to appear in the difference frame as darkened lines outlining a 
solid tumor mass. This difference frame can be frozen and stored to allow the surgeon 
to study the tumor image and identify tumor margins in real time during an operation. 
Moreover, the dye will remain for a longer period of time in tumor tissue compared to 
normal tissue. Therefore, after there is general appearance of the dye throughout the 

20 area of interest in both normal tissue and tumor tissue, the dye clearance in tumor tissue 
will be delayed, allowing another opportunity to visualize tumor margins by dye 
presence in tumor tissue but not in normal tissue. In more aggressive or malignant 
tumors, the higher the tumor grade, the longer the dye remains in tumor tissue. For 
lower grade or more benign tumors, the dye remains in tumor tissue for 45 sec to 2 min, 

25 whereas the dye can remain in more malignant tumors for up to 10 minutes. 

The inventive method is superior to established tumor imaging techniques, such 
as MRI (magnetic resonance imaging) because optical imaging can distinguish low 
grade tumors that cannot be distinguished with current MRI techniques (MRI is not an 
intraoperative technique) and updated images are continually available during a surgical 

30 procedure by readministering the dye. The dye ran be administered on multiple 
occasions during a surgical procedure after resection has begun to look at resected 
walls for residual tumor tissue. For CNS tumors, MRI techniques require an advanced 
stage tumor that has compromised the blood brain barrier to be able to image such a 
tumor. The present optical imaging method, by contrast, can image even low grade 

35 tumors that have not yet compromised the blood brain barrier. Therefore, optical 
imaging is a more sensitive technique than MRI, can be used intraoperatively, and 
provides an approximate means for grading tumors. 
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The dye can be any emr-absorbing dye that is safe for in vivo administration, and 
has a short half-life when administered intravenously or intraarterially. Example of such 
a dyes are indocyanine green, Photofiin®, NP e6 , BPD, Evans Blue, and combinations 
thereof Further, during surgical resection of a solid tumor, it is important that the dye 
5 be rapidly cleared from the area of interest. In that way, there can be repeated dye 
administrations to determine residual tumor tissue during resection. 

During an imaging study, it is important to continually update the averaged 
image frame to account for patient movement, particularly for an awake patient. This 
will account for circulating residual dye and patient movements or tissue movements 

10 due to surgical manipulation. 

For example, the inventive method for visualizing tumor tissue was utifized in a 
rat glioma model of the frontal lobe to test the ability of the inventive method to identify 
normal or edematous brain from tumor tissue and to determine if the inventive method 
could separate normal from abnormal tissue after all visible tumor had been resected. 

1 5 The dynamic nature of the dye's perfusion through the brain's vasculature allowed for 
differentiation of normal brain and tumor tissue. Moreover, with a spatial resolution of 
optical images below 20 umVpixel, even small areas of residual tumor can be identified. 
Further, tumor tissue was identified through intact rat skull, providing a method and a 
device for tumor imaging prior to surgery. 

20 Without being bound by theory, the dynamic differences in dye perfusion 

through normal brain tissue surrounding and tumor tissue could be accounted for by any 
one or a combination of the following four reasons: (I) greater extravasation of the dye 
through leaky tumor capillaries; (2) more rapid uptake of the dye by tumor tissue; (3) 
slower transit times through tumor tissue; and (4) preferential uptake of the dye by 

25 tumor cells. 

The rat glioma model has been examined and microvasculature compared to 
normal cortex. Blood flow in tumor tissue is slower and more variable than in nonnal 
tissue. The difference between tumor and nonnal brain have been attributed to tumor 
location, degree of infiltration, and necrosis. In other studies using cultured spheroids 

30 of C6 astroglial cells transplanted into rat brain, blood flow was slower in viable tumor 
than in nonnal rat brain. Microvessel volume fraction was equivalent between tumor 
and normal brain. However, since only about 50% of the tumor was actively perfused, 
the surface area of perfused mtcrovessels in the tumor was one-half that of the normal 
brain. These changes could account for a slower flow of dye through the tumor 

35 compared to normal brain and also lead to more rapid clearance by the normal brain in 
contrast to the tumor. 

The permeability of tumor capillaries is much higher than in nonnal brain. 
Leakiness of these capillaries leads to extravasation of larger particles resulting in 
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edema and an increase in interstitial pressure surrounding tumor microvessels. Since 
tumor microvessels do not contain norma! arteriole smooth muscle; they also have no 
local control of pressure gradients. This leads to a stasis of flow in tumor tissue. The 
overall effect on dye perfusion is longer transit times than in normal brain and an 
5 increase which prolongs duration of the large optical signal from tumor tissue. Such 
reasoning supports the dynamic changes in optical signal from tumor and normal brain 
that was seen during dye perfusion. There is nearly equivalent uptake, but a much 
slower transit time in tumor tissue resulting in prolonged increases in optical signal 
Also, tissue surrounding the tumor is expect to have increases in interstitial pressures 

10 but without leaky capillaries and other microvasculature changes, accounting for the 
fact that tumor tissue had an intermediate duration of optical changes. 

It is not dear whether a more rapid clearance mechanism of the dye from normal 
brain was occurring or if the dye was being preferentially sequestered by tumor cells. In 
the later case, hematoporphyrins are preferentially taken up into tumor cells, and 

15 accounts for the ability to use photodynamic therapy on such cells. If the dye remained 
completely intravascular, then a very uneven distribution of dye between normal and 
tumor cells would be expected. However, we observed the opposite from many 
intermediate images taken from areas between larger pial microvessels. 

The present invention further provides a method for imaging of cortical 

20 functional areas and dysfunctional areas, such as those areas of severe epileptic activity. 
The method involves administering a sensory signal for mapping a particular cortical 
fimction, or identifying an area of hyperactivity that is the location of epileptic activity 
in an epileptic patient. An epileptogenic area of the cortex will be visualized as 
spontaneously more active and can be imaged by the inventive apparatus using a 

25 method for mapping intrinsic signals of cortical activity. 

The method for visualizing intrinsic signals involves stimulating cortical tissue 
with specific paradigms. Various paradigms include, for example, presenting pictures of 
objects to a patient and asking the patient to name the abject to alter neuronal activity 
which will result in an associated intrinsic signal. 

30 Another feature of the inventive apparatus and method is the ability to image 

peripheral nerve damage and scarring. Nerves of the central and peripheral nervous 
system (PNS) are characterized by the ability to regenerate after damage. During 
operations to repair damaged peripheral or cranial nerves, one can image areas of nerve 
damage by imaging areas of blockage of intrinsic signals. For example, the nerve is 

35 exposed in the area of interest. The nerve is stimulated upstream of the site of damage. 
The active nerve pathway is imaged by intrinsic signals in the processed difference 
frame after activation. The site of nerve damage or blockage is evidenced by an abrupt 
end or diminution to the intrinsic signal at the damage site. In this way, the surgeon is 
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able to obtain real time information precisely where there is nerve damage and to 
correct the damage, if possible. 

Moreover, the inventive apparatus and ability to image intrinsic signals can be 
used when there is a need to remove tumor tissue that is located surrounding or 
5 adjacent to nerve tissue. For example a tumor called an acoustic neuroma is usually 
located surrounding an auditory (hearing) nerve. It is often a difficult procedure to 
remove tumor tissue without severing the auditory nerve (a cranial nerve) and causing 
one ear to become deaf or damage the facial nerve that innervates muscles that move 
the face. The inventive methods provide an ability to distinguish tumor tissue from 

10 surrounding nerve tissue using a dye. Additionally, the inventive method can 

continually provide information to the surgeon showing the precise location of the 
auditory or fecial nerve by continually or periodically stimulating the nerve with a sound 
paradigm for the auditory nerve, or backfiring the facial nerve from a facial muscle, and 
detecting the intrinsic signal associated with nerve activity. Accordingly, when there is 

15 tumor tissue in close proximity to nerve tissue, one can use both the ability to locate 
tumor tissue with a dye and to locate nerve tissue by detecting intrinsic signal using the 
same imaging apparatus. 

The imaging method can obtain information at the surface of an area of interest 
or can target an area of interest at a level deeper in tissue. Longer wavelengths of emr 

20 used to form the image (averaged control image and subsequent averaged images) can 
be used to probe areas of interest which are deeper into tissue. Moreover, if a 
difference image is created between the image seen with 500 nm emr and the image 
seen with 700 nm emr, the difference image will show an optical slice of tissue. 
Moreover, instead of using cutoff filters, administration of a dye can act as a tissue filter 

25 of emr to provide a filter in the area of interest. In this instance, it is desirable to utilize 
a dye that remains with tumor or normal tissue for a prolonged period of time. 

The present invention further comprises a method for enhancing sensitivity and 
contrast of the images obtained from tumor tissue or intrinsic signal difference images, 
comprising: (a) illuminating an area of interest with a plurality of wavelengths of emr, 

30 wherein there is at least a first wavelength of emr and a second wavelength of emr; (b) 
obtaining a series of frames corresponding to each wavelength of emr, wherein a first 
sequence of frames is from the first wavelength of emr, the second sequence of frames 
is from the second wavelength of emr and so on; (c) processing the first sequence of 
frames into a first averaged control image, the second sequence of frames into a second 

35 averaged control image and so on; (d) stimulating for intrinsic signals or administering a 
dye for tumor tissue imaging; (e) obtaining a first series of subsequent frames using the 
first wavelength of emr, a second series of subsequent frames using the second 
wavelength of emr, and so on, and processing the first, second and so on subsequent 
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series of frames into the first, second and so on subsequent averaged images, 
respectively; (f) obtaining a first difference image by subtracting the first averaged 
control image from the first subsequent averaged image and a second difference image 
by subtracting the second averaged control image from the second subsequent averaged 

5 image, and so on; and (g) obtaining an enhanced difference image by ratioing the first 
difference image to the second difference image. This can be accomplished, for 
example, with two single wavelength sources of emr> or by using a broad multiple 
wavelength source of emr and a plurality of longpass filters. Preferably, the 
monochromatic emr to illuminate the area of interest are from laser sources. 

10 The inventive apparatus and methods for imaging intrinsic signals and tumor 

tissue can operate outside of a surgical procedure setting. More specifically, it is 
possible to obtain tissue imaging through intact skin and bone. In some areas of the 
body longer wavelength visible light and near infrared emr can easily pass through such 
tissue for imaging, such as breast tissue. With dye injection, areas of increased 

15 vascularity, such as tumor tissue can be identified. 

Yet another aspect of the inventive method involves using an emr absorbing or 
fluorescent dye conjugated to a targeting molecule, such as an antibody, or more 
particularly, a monoclonal antibody or fragment thereof specific for an antigen surface 
marker of a tumor cell. The area of interest is illuminated with emr containing 

20 excitation wavelengths of the fluorescent dye but not emission wavelengths. This can 
be accomplished by use of a cutoff filter over the emr source. Preferably, the CCD 
camera is coupled to an image intensifier or micro channel plate (e.g. f KS-1381 Video 
Scope International, Wash DC) to increase the sensitivity of the system by several 
orders of magnitude and allow for visualization of cells having fluorescent dyes attached 

25 hereto. Examples of fluorescent dyes that can be conjugated to a targeting molecule 
include, for example, Cascade Blue, Texas Red and Lucifer Yellow CH from Molecular 
Probes Eugene OR. 

Still further applications of the inventive device are possible. For example, the 
device can be used for calibration of electrodes used to electrically stimulate the cortical 

30 surface (see, for example, Figures 1 and 2). A technique presently used by surgeons to 
map the functional organization of an awake patients cortex is to directly apply current 
(via stimulating-electrodes) to the surface of the cortex. The surgeon would like to 
apply the greatest intensity of stimulating current as possible without triggering an 
epileptic seizure or causing tissue damage. As a method of calibrating the stimulating- 

35 electrodes, the surgeon stimulates the cortex of the patient with currents of varying 
intensities and monitors the electrical activity by observing the output of recording- 
electrodes which have been placed directly on the surface of the patients brain. The 
surgeon applies current for several seconds with the stimulating-electrodes and checks 
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the output from the recording-electrodes for afterdischarge epileptifonn activity which 
may persist for a period of time after stimulation has ceased. The inventive device 
provides an accurate means of monitoring spatial extent of the cortex affected by the 
electrode stimulation current, and time course (if any) of stimulation-evoked activity 
5 persisting after cessation of the stimulation current. The method comprises acquiring a 
control image prior to stimulation, and then acquiring subsequent images during and 
after stimulation. The images are processed as described herein to provide a highly 
resolved spatial map of those areas of the cortex whose resting activity has been 
affected by the applied stimulating current. The inventive device provides a map of the 

1 0 spatial extent and timecourse of stimulation and epileptic activity, which the surgeon 
can use to choose an appropriate stimulating current for Ins or her electrodes. 

It is also possible to utilize the inventive device for simultaneous spatial mapping 
( : of dynamic changes of blood-flow in blood vessels and cortical activity using intrinsic 

optical changes, (see, for example, Figures 1 and 2) Without being bound by theory, 

1 5 the intrinsic optical changes within regions of larger blood vessels are due to a rate of 
change of the flow-rate within these vessels. The invention provides a method for 
monitoring these changes of flow within individual blood vessels. 

It is also possible to utilize the inventive device for a method to map functional 
organization of the cortex in an anesthetized patient during a neurosurgical procedure. 

20 (see, for example, Figure 5) A method comprises providing an afferent sensory 
stimulation to activate an area of cortex specific to processing this afferent sensory 
input, and using the evoked intrinsic signals observed with the inventive device and 
method to optically map regions of the cortex involved in the afferent sensory 
stimulation. 

25 For example;nduring surgical removal of a tumor, the surgeon may need to know 

which area of cortex is involved in processing sensory input from a limb in an 
/ anesthetized patient. The surgeon can apply an afferent sensory input, such as a 

vibrating stimulus, to the limb to cause the conduction of information to that part of the 
cortex involved in sensory processing for this limb. This sensory stimulus will activate 

30 the appropriate region of cortex which can then be mapped using the inventive device 
and method- Other examples of afferent sensory input include providing sound stimuli 
to activate auditory cortex, visual stimuli to map visual cortex, moving a joint, etc. This 
method is useful for mapping certain functional areas in an anesthetized patient. 

It is also possible to utilize the inventive device for a method to aid the surgeon 

35 intraoperatively in the acquisition of biopsies from a tumor margin, (see, for example, 
Figure 12). After the surgical removal of a tumor, the surgeon is left with the task of 
trying to distinguish what areas of the tumor resection margin contain remnants of 
tumor tissue. This is important since it is thought that in many tumor types, the 
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effectiveness of post-operative radiation and chemotherapy is correlated to the amount 
of tumor tissue left behind. The present technique of identifying these remnants of 
tumor is for the surgeon to remove small amounts of tissue in a random sampling 
fashion and said these samples for analysis to a pathologist This sampling and analysis 
5 must occur during the course of the surgery, since the surgeon needs this information to 
base his or her surgical decisions as to what tissue should be further resected. This 
technique suffers from several drawbacks, such as, the time required increases the 
length of the surgery thus increasing the cost and risk to the patient, and the random 
sampling method of determining biopsy sights is inevitably fraught with sampling error 
10 The present invention provides a method for quickly determining the likely sites of 
remnant tumor tissue within a tumor margin. This can assist a surgical decision as to 
what areas of tissue should be sampled for biopsy purposes and which areas should be 
immediately removed a likely containing tumor tissue. 

I* EXAMPLE 1 

This example illustrates optical changes on a human subject by direct cortical 
electrical stimulation. Surface electrical recordings (surface EEG, ECOG) were 
correlated with optical changes. Figures 1 and 2 illustrate dynamic changes in intrinsic 
optical properties of human cortex during direct electrical stimulation of the cortex and 

20 during epileptiform activity. Figures A, B, C, and D, exemplify a typical optical 

imaging study where the inventive device is utilized to provide dynamic information 
with high spatial resolution of the intrinsic optical changes of the human cortex during 
either awake or anesthetized neurological surgical procedures. In Figure 1, intrinsic 
optical changes ware evoked in an awake patient during stimulating-electrode 

25 "calibration' 1 . Four stimulation trials were sequentially applied to the cortical surface, 
each stimulation evoldrig an epileptiform afterdischage episode. A stimulation trial 
consists of: 1) monitoring resting cortical activity by observing the output of the 
recording electrodes for a brief period of time, 2)applying and electric current via the 
stimulation-electrodes to the cortical surface at a particular current for several seconds, 

30 and 3) monitoring the output of the recording electrodes for a period of time after 
stimulation has ceased. 

A series of images (each image consisting of an average of 128 frames acquired 
at 30 Hz) were acquired during each of the four stimulation trials. A current of 6 raA 
was used for the first three stimulation trials, and 8 mA for the fourth. After a sequence 

35 of 3-6 averaged control images were acquired, a bipolar cortical stimulation current 
was applied (either 6mA or 8 mA) until epileptiform after discharge activity was evoked 
(as recorded by the surface electrode). Images were continuously acquired throughout 
each of the four stimulation trials. 
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The percentage change in absorption of light for each pixel was calculated for 
each image acquired during the four stimulation trials. The average percentage changes 
over the four areas (indicated by the four square regions marked in Figure 1 A) were 
plotted graphically in Figures IB, 1C, and ID for comparison and analysis of the 
5 dynamic changes occurring in these four spatial areas. 

Figure 1 A is a view of a human cortex just anterior to face-motor cortex with 
one recording electrode (r) and two stimulating electrodes (s), and four sites (the boxed 
areas labeled 1, 2, 3, and 4) where the average percent changes of absorption over these 
areas were determined. The cortex was illuminated with emr >690 nm. The scale bar is 
10 1 cm. 

Figure IB are plots of the percent change of emr absorption per second in the 
spatial regions of boxes 1 and 3 (as labeled in Figure 1A). For both regions, the peak 
change is during the fourth stimulation trial (at 8 mA) in which the greatest amount of 
stimulating current had induced the most prolonged epileptiform afterdischarge activity. 

15 The changes within box 3 were greater and more prolonged than those of box 1 . Box 3 
was overlying the area of the epileptic focus (the excitable area of tissue possibly 
respons&le for the patients epilepsy). 

Figure 1C show plots of the percent change of emr absorption per second in the 
spatial regions of boxes 1 and 4 (as labeled in Figure 1 A). Box 1 overlays and area of 

20 cortical tissue between the two stimulating electrodes, and box 4 overlays a blood 

vessel. The changes within box 4 are much larger and in the opposite direction of box 
1, and also that these changes are graded with the magnitude of stimulating current and 
afterdischarge activity. Since the changes in box 4 are most likely due to changes of the 
blood-flow rate within a blood vessel, this plot shows mat the Invention can 

25 simultaneously monitor cortical activity and blood-flow. 

Figure ID shows plots of the percent change of emr absorption per second in 
the spatial regions of boxes 1 and 2 (as labeled in Figure 1 A). Note that although these 
two areas a nearby each other, their optical changes are in the opposite direction during 
the first three stimulation trials using 6 mA current. The negative going changes within 

30 the region of box 2 indicate that the invention may be used to monitor inhibition of 
cortical activity as well as excitation. 

AD imaging procedures reported in this example and the patient consent form 
were reviewed and approved by the University of Washington Human Subjects Review 
Committee. All patients signed an informed consent statement for both the surgery and 

35 the imaging experiments. The cortex was evenly illuminated by a fiberoptic emr passing 
through a beam splitter, controlled by a D.C. regulated power supply (Lambda, Inc.) 
and passed through a 695 nm longpass filter. Images were acquired with a CCD 
camera (COHU 6500) fitted to the operating microscope with a specially modified 
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cmeadaptor. The cortex was stabilized with a glass footplate. Images were acquired at 
30 Hz and digitized at 8 bits (512 x 480 pixels, using an Imaging Technology Inc. 
Series 151 system, Woburn MA). Geometrical transformations were applied to images 
to compensate for small amounts of patient motion (Woblberg, Digital Imaging 

5 Warping* IJE.E.E. Computer Society, Los Alamitos, CA, 1988). Subtraction of images 
collected during the stimulated state (e.g., during cortical surface stimulation, tongue 
movement, or naming) from those collected during a control state with subsequent 
division by a control image resulted in percentage difference maps. Raw data (/.e., no 
digital enhancement) were used for determining the average optical change in specified 

10 regions (average sized boxes was 30 x 30 pixels or 150-250 \ixrP). For pseudocolor 
images, a linear low pass filter removed high frequency noise and linear histogram 
transformations were applied. Noise was defined as the standard deviation of 
fluctuations in sequentially acquired control images as 0.003-0.009. 

The optical changes between the stimulating electrodes (ate #1 in Fig. 1 A) and 

15 near the recording electrode (ate #3) showed a graded response to the intensity and 
duration of each afterdischarge episode (Figure IB). The spatial extent of the 
epileptiform activity was demonstrated by comparing a baseline image collected before 
stimulation to those obtained immediately after stimulation. The intensity and spread of 
the optica] changes were much less following stimulation #2 (shortest least intense 

20 afterdischarge episode) than after stimulation #4 (longest most intense afterdischarge 
episode). 

When the optical changes were below baseline, the surface EEG recordings did 
not identify epileptiform activity (n « 3 patients) At site #3 in Figure 2A1, the optical 
changes after stimulation were below baseline (i.e., black regions in Figure 2A3). 
25 However, during the fourth stimulation, the epileptiform activity spread into the area of 
site #3 and the optical signal did not go below baseline until later (site #3, Figure IB). 
This negative optical signal likely represents inhibited neuronal populations (an epileptic 
inhibitory surround), decreased oxygen delivery, or blood volume shunted to activated 
regions. 

30 Figure 3 shows a sequence of dynamic changes of optical signals identifying 

active areas and seizure foci. This Figure shows eight percentage difference images 
from the stimulation trial 2 described in the previous two Figures. Each image captures 
a two second interval. Note the focal area of greatest optical change in the center of 
images 4, 5, and 6 that indicates the region of greatest cortical activity. The magnitude 

35 of optical change is indicated by the gray-scale bar in Figure 2. Each image maps an 
area of cortex approximately 4 cm by 4 cm. 

Figure 4 illustrates a real-time sequence of dynamic changes of stimulation* 
evoked optical changes in human cortex. Figure 4, panels 1 through 8, show eight 
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consecutive percentage difference images, each image is an average of 8 frames (< 1/4 
second per image). Each image maps to an area of cortex that is approximately 4 cm by 
4 cm. 

In Figure 6, stimulation mapping of the cortical surface was performed on 
5 awake patients under local anesthesia to identify sensory/motor cortex and Broca's 
areas. During three "tongue wiggling*' trials images were averaged (32 frames, 1 sec) 
and stored every 2 seconds. A tongue wiggling trial consisted of acquiring 5-6 images 
during rest, then acquiring images during the 40 seconds that the patient was required 
to wiggle his tongue against the roof of his mouth, and then to continue acquiring 

1 0 images during a recovery period. The same patient was then required to engage in a 
"language naming" trial. A language naming trial consisted of acquiring 5-8 images 
during rest.(control images - the patient silently viewing a series of blank slides), then 
acquiring images during the period of time that the patient engaged in the naming 
paradigm (naming a series of objects presented with a slide projector every 2 seconds, 

1 5 selected to evoke a large response io Broca's area), and finally a series of images during 
the recovery period following the time when the patient ceased his naming task (again 
viewing blank slides while remaining silent). Images Al and Bl are gray-scale images 
of an area of human cortex with left being anterior, right-posterior, top-superior, and 
the Sylvan fissure on the bottom. The two asterisks on Al, Bl, A2, and B2 serve as 

20 reference points between these images. The scale bars in the lower right corner of Al 
and Bl are equal to 1 cm. In Al, the numbered boxes represent sites where cortical 
stimulation with electrical stimulating electrodes evoked palate tingling (1), tongue 
tingling (2), speech arrest-Broca's areas (3,4) and no response (11,12,17, 5-premotor). 
Image A2 is a percentage difference control image of the cortex during rest in one of 

25 the tongue wiggling trials. The gray-scale bar on the right of A2 shows the relative 
magnitude of the gray scale associated with images A2, A3, B2 and B3. Image A3 is a 
percentage difference map of the peak optical changes occurring during one of the 
tongue wiggling trials. Note that areas identified as tongue and palate sensory areas by 
cortical stimulation show a large positive change. The suppression of baseline noise in 

30 surrounding areas indicate that, during the tongue wiggling trials, language-motor areas 
showed a negative going signal. Image B2 is a percentage difference control image of 
the cortex during one of the language naming trials. Image B3 is a percentage 
difference image of the peak optical change in the cortex during the language naming 
task. Large positive going signals are present in Broca's area, however negative going . . 

35 signals are present in tongue and palate sensory areas. 

Figure 7 shows a time course and magnitude plot of the dynamic optical changes 
in human cortex evoked in tongue and palate sensory areas and in Broca's area 
(language). Figure 7 shows the plots of the percentage change in the optical absorption 
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of the tissue within the boxed regions shown in Figure 6, images Al and B 1, during 
each of the three tongue wiggling trials and one of the language naming trials. Diagram 
7A shows the plots during the three tongue wiggling trials averaged spatially within the 
boxes 1, 2, 3, and 4 as identified in Figure 6, image Al. Diagram 7B shows the plots 

5 during one of the language naming trials averaged spatially within the boxes 1-7 and 1 7. 
These results agree with those data reported by Lee et al. {Amu Neurol 20:32, 1986) 
who reported large electrical potentials in the sensory cortex during finger movement 
The magnitude of the optical changes in the sensory cortex during tongue movement 
(10-30%) parallels sensory/motor cortex studies where cerebral blood flow increases 

10 10-30% during motor tasks (Cdlebatch et al., J, Neurophysiol 65: 1392, 1991). 
Further, utilizing Magnetic Resonance Imaging (MRI) of blood volume changes in 
human visual cortex during visual stimulation, investigators have demonstrated 
increases of up to 30% in cerebral blood volume (Belliveau et aU Science 254:716, 
1991). 

15 Optical images were obtained from this same cortical region (i.e., area of 

interest) while the patient viewed blank slides and while naming objects on slides 
presented every two seconds. Percentage difference maps obtained during naming 
showed activation of the premotor area. The sites of speech arrest and palate tingling 
were identified by surface stimulation and demonstrate optical signals going in the 

20 opposite direction. The area of activation was clearly different from that evoked by 
tongue movement without speech production, The optical images of premotor cortex 
activation during naming were in similar locations to the cortical areas identified in PET 
singje word processing studies (Peterson et aL, Nature 331:585, 1991; and Frith et al., 
J, Neuropsychologic? 29: 1 137, 1991). The optical changes were greatest in the area of 

25 the cortex traditionaliydefined as Broca's area and not in areas where electrical 
stimulation caused speech arrest. 

Figure 8 illustrates an optica! map of a cortical area important for language 
comprehension (Wernicke's area) in an awake human. Figure 8 image A shows the 
cortical surface of a patient where the anatomical orientation is left-anterior, bottom- 

30 inferior, with the Sylvan fissure running along the top. After optical imaging, all 

cortical tissue to the left of the thick black line was surgically removed. Sites #1 and #2 
were identified as essential for speech cortical stimulation blocked ability of 
subject to name objects). At site #3, one naming error in 3 stimulation trials was found. 
As the surgical removal reached the area labeled by the asterisks on the thick line, the 

35 patient's language deteriorated. All the unlabeled sites in Figure 8 A had no errors while 
naming slides during cortical stimulation. Figure 8, image B shows an overlay of a 
percentage difference image over the gray-scale image of the cortex acquired during a 
language naming trial (see description for Figure 6 for description of the language 
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naming trial). The magnitude of the optical change is shown in the gray-scale bar on 
the lower right of the image. This image demonstrates how a surgeon might use this 
invention intraoperatrvely to map language cortex. 

Figure 9 illustrates a time course and magnitude of dynamic optical changes in 
5 human cortex evoked in Wernicke's area (language comprehension). Figure 9A shows 
plots of percentage change in optical absorption of tissue within the boxed regions 
shown in Figure 8. Figure 9A shows plots of boxes 1 and 2 overlay essential language 
sites, and boxes labeled 4, 5, and 6 overlay secondary language sites. Each of these five 
sights showed significant changes occurring while the patient was engaged in a 
10 language naming task. Figure 9B show percentage changes from the six unlabeled 

boxes shown in Figure 8. There were no significant increases or decreases within these 
anterior sites. These data demonstrate that optical imaging can also identify both 
essential and secondary language areas that must be preserved during neurosurgical 
procedures. 

15 

EXAMPLE 1 

This example illustrates indocyanine green imaging of a low grade tumor. A 
MRI scan was conducted before the operation. Additionally, the patient was 
investigated for tumor tissue using the apparatus described according to the invention 

20 and specifically used in Example 1 . 

An averaged control image was obtained of the particular cortical surface area 
of interest. Indocyanine green dye was administered into a peripheral intravenous 
catheter as a bolus at time 0. Figure 1 0 illustrates differential dynamics of dye to 
identify low grade human CNS tumor. This series of images are from a patient having a 

25 low grade CNS tumor (astrocytoma, grade 1). In Figure 10A (upper left) ihe lettered 
labels placed upon the brain by the surgeon overlay the tumor as identified 
intraoperatrvely by ultrasound. However, tumors of this type and grade are notoriously 
difficult to distinguish from normal tissue once the surgical removal of the tumor has 
begun. Figure 10B (middle left) shows a difference image taken approximately 15 

30 seconds after intravenous injection of dye (indocyanine green at 1 mg/kg). Figure 10C 
(lower left) shows the difference image about 30 seconds after dye administration. The 
area of the tumor tissue showed the first tissue staining. Figure 10D (top right) shows 
that in this low grade tumor, all tissue (both normal and abnormal) showed staining at 
45 sec after dye administration. Figure 10E (middle right) is one minute after dye 

35 administration and Figure 10F is five minutes after dye administration (showing 

complete clearance in this low grade tumor). These data show that indocyanine green 
enters tow grade tumor tissue faster than normal brain tissue, and may take longer to be 
cleared from benign tumor tissue than normal tissue. Therefore, it is possible to image 
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even low grade tumors with this apparatus; Furthermore, it is possible to distinguish 
intraoperatively low grade tumor tissue from surrounding norma! tissue. 

Therefore, it is possible to image even low grade tumors by the inventive 
apparatus. Subsequent pathology of this tumor tissue established it as a low grade 
5 glioma. 



EXAMPLE 3 

This example illustrates the image of a highly malignant CNS tumor 
(glioblastoma). A patient was imaged in a neurosurgical procedure as described in 

1 0 Example 1 . The tumor imaging procedure was the same as in Example 2. The series of 
images in Figure 1 1 are from the cortex of a patient with a malignant CNS tumor 
(glioblastoma; astrocytoma, Grade IV). Figure 1 1 A (upper left) shows a gray-scale 
image in which malignant brain tumor tissue was densest in the center and to the right 
but elsewhere was mostly normal tissue (as was shown by pathology slides available one 

15 week after surgery). Figure 1 IB (middle left) is the difference image at 15 seconds 

after intravenous injection of indocyanine green, showing the dynamics of dye perfusion 
in the first seconds in malignant tissue are similar to those in the first few seconds of 
benign tumor tissue (see Figure 1 1C). Figure 1 1C (lower left) shows that at 30 seconds 
the malignant tissue is even more intense by comparison to the normal tissue. Figure 

20 1 ID (upper right, 1 minute after dye injection) and 1 IE (lower right, 10 minutes after 
dye injection) show that unlike benign tumor tissue, in malignant tumor tissue, dye is 
retained significantly longer, and in some cases, continues to sequester in the malignant 
tumor tissue over longer periods of time. Therefore, it is possible with this device to 
identify malignant tumor tissue, distinguish intraoperatively between normal and 

25 malignant tumor tissue, and to distinguish between the various grades of tumor (e,g M 
normal vs. benign vs. malignant). Thus, it is possible to not only image the location of 
tumor tissue, but also to grade the tumor with more malignant tumors retaining dye for 
a longer period of time than a lower grade tumor. 

30 EXAMPLE 4 

This example illustrates a series of images taken after resection of a malignant 
CNS tumor until the tissue appeared to be normal This type of imaging of tumor 
margins provides a novel method of real-time imaging of tumor margins. After 
resection, the surgeon performed multiple histological margin sampling and when 

35 waiting for frozen section results, the images shown in Figure 1 3 were obtained. Figure 
13 shows a series of images and difference images of an area of interest where a tumor 
was resected. The area of interest was thought to be free of tumor tissue after the 
surgical removal of the tumor. Normally, in this size of a resection margin, only a single 
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frozen sample would be taken for pathology analysis. For the purpose of this study, 
five biopsies were taken from the margin to aid in correlating the histology with the map 
obtained by the invention. Figure 12A (top left) shows a gray-scale image of the tumor 
margin. Figure 12B shows the margin with labels that the surgeon placed directly on 
5 brain. The purpose of these labels were to identify where the surgeon was going to 
remove biopsy samples for histological analysis after difference images were acquired 
with the inventive device. Kgure 12C (tower left) shows the difference image 1 minute 
after intravenous injection of dye and Figure 12D (lower right) shows the difference 
image 10 minutes after dye injection. These post-dye difference images reveal a number 

10 of sights that contain tumor tissue as well as areas of normal tissue, The accuracy of 
the optical imaging was confirmed post operatively by analysis of the biopsies. Note 
that a small area on the lower right of Figure 12D indicates a possible region of tumor 
tissue that would not have been biopsied by the surgeon. Hence, even in the case of 
extensive biopsy, the sampling error exceeds the accuracy of the invention. These data 

15 show that the invention is able to identify small remnants of tumor tissue in a tumor 
margin after resection of a tumor. As well, the invention could act as an aid to 
removing biopsies from the site of a tumor margin and reduce the innate sampling error 
associate with the presently used random sampling technique. 



20 EXAMPLE 5 

This example illustrates a means for setting the CCD to optimize the apparatus 
to be able to detect signal with maximum sensitivity across a full dynamic range. The 
CPU should be programmed with software having the following features: (1) an output- 
analog signal, values of the image are close to saturating on the bright end (i.e. , close to 

25 225) are displayed as a distinct color, such as red; (2) values that are close to the dark 
end (/.e,, are close to zero) are also displayed as a distinct color, such as blue. The 
following procedure is an example of an adjustment of the CCD camera. 

1 . With the gain and black-level on a camera-control box (CCB) initially set 
to 0, increase the emr intensity until the video signal is just saturating on the bright-end 

30 (/.<?., some values in the output-analog signal can be seen to be close to 255). 

2. In crease the blade-level on the CCB until the output image can be seen 
to be saturating on the dark end (/.*., some values in the output analog image can be 
seen to be close to 0). 

3 . Increase the gain on the CCB until some values of the output analog 
35 image can be seen to be saturating on the high end. 

4. Iterate steps (2) and (3) until either (a) the gain is set to its maximum 
possible value, or (b) the black4evel is set to its maximum possible value, or (c) the 
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image is maximally enhanced across is full dynamic range (that is, no further 
adjustments of CCB gain, black-level or emr source will improve the image. 

5. If in step (4) (a), the gain is set to its maximum level, or (b) the black- 
level is set to its maximum level, but the output image is still not maximally enhanced, 
5 then in the case of (a), decrease the setting on the CCB gam slightly, increase the emr 
source intensity until just saturating the bright end, and return to step (2). In the case of 
(b), decrease the setting of the black-level slightly, decrease the emr intensity, and 
return to step (3). 

10 EXAMPLE 6 

This example illustrates a series of experiments using a rat glioma model 
imraoperativeiy to investigate whether the inventive method and inventive device could 
function in an operating room setting to provide real time information to the surgeon 
regarding resection of all tumor tissue. The rat glioma model is a standard predictive 

1 5 model and was used to delineate dye uptake, clearance and overall parameters of optical 
imaging that result in the best images. The advantages of tins model are the ability to 
consistently get reproducible tumors for imaging studies and to be able to rescect tumor 
under an operating microscope and still find residual tumor with the inventive optical 
imaging. A disadvantage of this model is the more sarcoma-like appearance of the 

20 tumor and a lesser degree of vascularity compared to human gliomas. 

Briefly, the rat glioma model uses an ethyliutrosourea-induced F-344 rat tumor 
line developed from a clonal population of a spinal malignant astrocytoma. This tumor 
is similar to human astrocytomas microscopically and m vrvo 9 because both have 
stellate-shaped cells in the brain parenchyma and both have introcytoplasmic filaments 

25 80-100 iim in diameter as seen by scanning electron microscopy. The glioma cells were 
maintained to Weymouth's medium supplemented with 10% fetal calf serum. Viable 
cells (5 x 10 4 ) were trypsinized from a monolayer culture and implanted stereotaxically 
into the right cerebral hemisphere of 30 syngeneic female rats, each weighing 140-160 
g. The stereotaxic coordinates ft>r right frontal lobe implantation were 4.5 mm anterior 

30 to the frontal zero plane, 3 mm right3 mm right from the midline and 6 mm deep. The rats were 
anesthetized for implantation. The heads were shaved and scalps opened, and a 1 mm 
burr hole made at the appropriate coordinates. The cells were injected through a 27 
gauge needle, the needle left in place for 30 sec post injection and the hole was covered 
with bone wax. The scalp was sutured and the animals observed for 3-4 hrs until they ... 

35 returned to normal activity and feeding. The animals were used 10-14 days after tumor 
implantation. In this model, animals begin to show clinical symptoms from the tumor by 
16*1 9 days, such as decreased activity and feeding, hemiparesis and eventually succumb 
between 19-27 days from mass effects due to tumor expansion. 
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Fourteen animals underwent complete study, including imaging before and after 
resection of the tumor. For study, the animals were anesthetized with 2% isoflurane, 
and the femoral vein canulated for administration of the dye. Anesthesa was 
maintained with ot-cMoralsoe (50 mg/kg administered ip) and urethane (160 mg/kg 
5 administered ip). The animals were placed in a stereotaxic holder. Imaging studies 
were then carried out before (see example 7 below) or after removal of the cranium. 
The tumor typically occupied the anterior one half to two thirds of the right hemisphere 
exposure. The compressed brain without any tumor infiltration was defined as the 
tumor surround to separate it from the normal hemisphere on the contralateral side. 
10 Indocyanine green was used as the intravenous dye. No dye was found in the 
cerebrospinal fluid after administration. 

The cortical surface was first imaged, and then an operating microscope was 
used to attempt gross total removal of the tumor. Sites were then chosen for biopsy 
based on optical imaging results and later analyzed histologically. The biopsy 
15 specimens were fixed in 1 0% paraformaldehyde, Nissl stained and mounted. All 

specimens were read blindly and labeled either positive or negative for tumor. These 
data were correlated to the optical imaging results to identify residual tumor and 
statistical analysis (Chi square or student West) performed to determine the significance 
of the results. 

20 T* 16 imaging apparatus is described herein. Light was from a tungsten-halogen 

bulb regulated by a D.C. power supply, passed through a Iongpass filter (690 nm), and 
through a right angled prism reflected through a 50 or 1 00 mm objective lens onto the 
cortical surface. The reflected light was collected by the same objective lens and 
focused by a projection lens onto the surface of a CCD camera (COHU 6300). The 

25 imaging apparatus was attached to the stereotaxic frame which was rigidly fixed to a 
vibration isolation table. Specially designed automatic warping algorithms were 
designed to compensate for small amounts of movement. Images (512 x 480 pixels) 
were acquired at 30 Hz and digitized at 8 bits (256 gray levels). Every 2 sec, a single 
image comprising 30 averaged frames was collected (1 sec) and then stored (1 sec). 

30 Control images were collected prior to injection of the dye and then for 2 min after dye 
injection. The dye injection was made over a 1 sec period while the last control image 
was being stored. A period of 20 min was allowed between dye injections to allow 
optical images to return to baseline. The initial control images of each trial were 
subtracted from each other to insure that the baseline starting point of each trial was 

35 equivalent. 

A single control image was chosen and then subtracted from each of the 
controls (4*6 images) and each of the post-dye injection images. The resultant image 
was divided by the original control image and multiplied by 100 to give a composite 
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percentage difference for the entire sequence before and after dye injection. The optical 
change that occurred between separate control images was 0.2-0.7%, whereas the peak 
changes resulting from dye injection were in the range of 5-40%. The control 
percentage difference image are represented in the attached figures. The spatial 
5 resolution of an individual pixel in the image ranged from 13.5 x 1 1 .7 nm3i%e2Sfi-2ffi%. The contr 
UnA Boxes measuring from 15-30 pixels per side were drawn on the images. The 
average percentage change in the individual boxes was calculated and used to 
demonstrate graphically the optical changes over time in the different types of tissue. 
Imaging studies were performed on fourteen animals. The time course of dye 

1 0 perfusion through the tissue had a dynamic aspect. Optical imaging of indocyanine 

green dye perfusion at a dose of 1 mgfltg in 16 separate runs from a cortical surface in 9 
different animals demonstrated the dynamic nature of the optical changes. Figure 16 
show a sequence of images to illustrate the dynamic differences of dye absorption 
changes between tumor and non-tumor tissue. Figure 16A shows a gray-scale image of 

1 5 the area of interest. This is the same animal as shown in Figure 1 6, however the 
cranium has now been removed so as to expose the left hemisphere containing the 
glioma, and the right hemisphere containing normal tissue. Box 1 overlays the tumor, 
Box 2 the tumor-surround, and Box 3 overlays normal tissue. Figure 16B shows the 
difference image of the area of interested 1 second after 1 mg/kg of indocyanine green 

20 had been intravenously injected into the animal. During this initial time, the tumor 
tissue is the first to show a measurable optical change indicating the uptake of dye 
occurs first in the tumor tissue. The gray-scale bar indicate the relative magnitude of 
the optical changes in the sequence of difference images. Figures 16C and 16D show 
difference images of the area of interest 4 seconds and 30 seconds respectively after dye 

25 injection. At these intermediate stages dye appears to collect in both normal and tumor 
tissue. Figures 16E and 16F show difference images of the area of interest 1 minute and 
5 minutes respectively after injection of dye. At these late: times, it becomes dear that 
dye is still collecting in tumor tissue even thought it is being cleared from normal tissue. 
The peak optical changes occur after 6 seconds from dye administration, but 

30 after the normal hemisphere begins to clear the dye, the tumor tissue continues to 
maintain a large optical due to a lack of dye clearance. These changes localize 
anatomically to the tumor ate. 

The optical signals begin to change within the first 2-3 seconds after dye 
injection and peak 6 seconds after injection in all three areas, tumor tissue, tumor 

35 surround and normal brain. However, the three Afferent tissue types are differentiated 
by the rate of rise over the first four seconds, the peak optical change reached, and the 
eventual plateau that occurs after the first 30 seconds. The tumor tissue had a 
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significantly different peak percentage difference change (40.5 ± 9.6%) than either the 
tumor surround (1 6.4 ± 6.8%) or the normal brain (9.7 ± 4.7%). 

Figure 17 is a plot of an average of the percentage change in ernr absorption 
average over the spatial areas indicated by boxes 1, 2, and 3 from Figure 1 6A. The 
5 increase in absorption is a function of the concentration of dye in the tissue at a 
particular time. The graph labeled "tumor tissue" is a plot of the dynamics of the 
absorption changes within box 1 from Figure 16A. The graph labeled "tumor surround* 1 
is a plot of the dynamics of the absorption changes within box 2 from Figure 16A. The 
graph labeled "normal brain" is a plot of the dynamics of the absorption changes within 

10 box 3 from 16A. This data as well as that from Figure 16 show that the inventive 

method and device is able to distinguish not only tumor from non-tumor tissue, but also 
tumor-surround areas which contain varying densities of tumor vs. normal cells. 

Since the peak optical change was always reached 4-6 seconds after dye 
injection, there was also a significantly faster rate of optical change in the tumor tissue 

1 5 compared to the tumor surround or the normal brain. A more rapid onset of dye 

perfusion into the tumor tissue was displayed as a faster time course. The tumor tissue 
had a more rapid and greater rise time than either the tumor surround or normal brain (p 
<0.01). 

An essential feature to be able to differentiate normal from abnormal tissue is 
20 distribution of the dye into three very different tissue compartments. In 13 of 14 

animals there was a prolonged increase (>2 min) in the optical signal in the tumor after 
the normal and tumor surround tissue had returned to baseline. Finally, even the normal 
and tumor surround tissue were significantly different in dye uptake (rise time; normal 
2.4%/sec; tumor surround 4.0%/sec). Therefore, the dynamic features of dye uptake 
25 and clearance are critical for determining the type of tissue involved in imaging 
resection margins. 

The rat glioma model also provided an opportunity to look at imaging resection 
margins once all visible tumor had been removed. Figure 1 8 A shows a higher 
magnification image of the left hemisphere tumor margin of the animal after the tumor 

30 had been resected. Boxes 1 are over areas that contained small traces of residual tumor 
cells, and boxes 2 are over areas that contained only normal tissue. The gray-scale bar 
indicates the magnitude of optical change in the difference images. Figures 18B, 18C, 
and 18D show difference images of the tumor margin 4, 30, and 60 seconds after 
intravenous dye injection respectively. Minute biopsies were taken from areas that 

35 showed preferred dye containment and from areas from which the dye cleared rapidly. 
These biopsies were analyzed blindly and later correlated to the location from which the 
biopsies were taken. Those biopsies taken from areas which cleared dye were shown to 
contain only normal cells, whereas biopsies taken from areas which sequestered dye 
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were shown to contain tumor cells. The more rapid rate of rise seen in cortical surface 
imaging was still present for the resection margins that were positive for tumor 
compared to normal brain* Again, significant differences between the tumor and the 
normal brain existed for the rate of rise, peak optical change, and plateau 60 seconds 

5 after dye injection (all p<0.01). Figures 15-18 demonstrate that the inventive method 
and device can be used in combination with multiple injections of dye for repeated 
application throughout a tumor resection surgery (in this case, 4 separate injections of 
dye were given). Furthermore, extremely small islands of residual tumor can be mapped 
within the tumor margins. 

10 Sensitivity and specificity of optical imaging was determined for 34 samples (n = 

12 animals) Of 15 biopsy sites deemed negative for tumor by optical imaging, 14 of 15 
were clear of tumor by histological analysis (sensitivity 93%). Most of the specimens 
that were negative for tumor were taken from the posterior wall of the tumor resection 
cavity or the depth of the cavity (where the hippicampus or denate gyrus were 

1 5 frequently biopsied). Of 1 9 biopsy sites deemed positive for tumor by optical imaging, 
1 7 of the biopsy specimens were read as positive for tumor (specificity 89.5%). The 
two sites that were negative for tumor on histology but positive for tumor by optical 
imaging had increased cellularity but were deemed negative for tumor because there 
was no focus of tumor tissue present The overall significance of these results are p < 

20 O.00L 

Figure 19 shows dynamic information of dye uptake and clearance in tumor vs. 
non-tumor tissue. This is a plot of an average of the percentage change in emr 
absorption average over the spatial areas indicated by boxes 1 and 2 from Figure 18 A. 
The increase in absorption is a function of the concentration of dye in the tissue at a 

25 particular time. The graph labeled "margins tumor n is a plot of the dynamics of the 

absorption changes within box 1 from Figure 18 A. The graph labeled "margins normal" 
is a plot of the dynamics of the absorption changes within box 2 from Figure 18A. This 
data as well as that from Figure 19 show that the inventive device and method are able 
to distinguish tumor from non-tumor tissue within tumor margins with extremely high 

30 spatial and temporal resolution. 

EXAMPLE 7 

This example illustrates a series of experiments using a rat glioma model 
through an mtact skull to investigate whether the inventive method and inventive device 
35 could function in to image tumor tissue through an intact skull and through intact skin 
prior to or after surgeiy. Far red wavelengths of emr are known to penetrate through 
bone and skin. Imaging of tumor tissue was attempted through the intact skull of the 
rat. The extent of tumor identified was not as accurate as with the cortex exposed, 
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however, the area lying beneath the skull with tumor tissue was easily identified, 
localized and continued to concentrate dye after several minutes. Initially, after dye 
injection, the area of the tumor demonstrated a much larger signal than the normal brain 
of the contralateral hemisphere. One minute after dye injection, the dye had been 
5 cleared from the normal brain and the only residual signal remained in tumor tissue and 
the sagital/transverse sinuses. 

Figure 14A is a gray-scale image of the cranial surface of a rat. The sagital 
suture runs down the center of the image. Tumor cells had been injected into the left 
side some days earlier so that this animal had developed a glioma on the left hemisphere 

10 of its brain. The right hemisphere was normal. Box 1 lays over the suspect region of 
brain tumor, and box 2 lays over normal tissue. Figure 14B is a difference image 1 
second after indocyanine green dye had been intravenously injected into the animal. 
The region containing tumor tissue becomes immediately viable through the intact 
cranium. Figure 14C shows that 5 seconds after dye injection the dye can be seen to 

1 5 profuse through both normal and tumor tissue. Figure 14D shows that 1 minute after 
dye injection, the normal tissue has cleared the dye, but dye is still retained in the tumor 
region. The concentration of dye in the center of this difference image is dye circulating 
in the sagital anus. 

The time course of optical changes imaged through the cranium from ten runs in 
20 four animals are shown in Figure 1 5. The optical changes were determined by the 
average optical change in a box placed directly over the tumor and over the normal 
hemisphere. The increase in absorption is a function of the concentration of dye in the 
tissue at a particular time. The graph labeled "extracranial tumor" is a plot of the 
dynamics of the absorption changes within box 1 from Figure 14A. The graph labeled 
25 "extracranial: normal" is a plot of the dynamics of the absorption change within box 2 
from Figure 14 A. The peak optical changes for the tumor imaged through the cranium 
were 13.1 =fc 3.9% and this was significantly greater compared to normal brain of 7.8 =fc 
2.3% (p < 0.01 ). The plateau phase 60 seconds after dye injection was also significantly 
greater in tumor tissue (40.5 ± 9.6%) compared to normal brain (3.1 * 0.7%) (p < 
30 0.01). 

EXAMPLES 

This example illustrates a rat model showing activation of sensory cortex by 
stimulation of a peripheral nerve. More specifically, afferent sensoiy input was 
35 generated in an anesthetized rat by directly stimulating the sciatic nerve. In Figure 5, 
the leftmost image is a gray-scale image of hind limb somatosensory cortex in an 
anesthetized rat. The magnification is sufficiently high so that individual capillaries can 
be distinguished (the smallest vessels visible in this image). The center image is an 
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image of a percentage difference control optical image during rest. The magnitude of 
optical change is indicated by the gray-scale bar in the center of this image. The arrow 
beside this gray-scale indicates the direction of increasing amplitude. The rightmost 
image is a percentage difference map of the optical changes in the hind limb sensory 
5 cortex during stimulation of the sciatic nerve. Therefore, it is possible to utilize the 
inventive device and method to map functional areas of the cortex corresponding to 
different areas of the body. 

EXAMPLE 9 

10 This example illustrates that differential dynamics of dye uptake and retention 

can characterize and identify tumor tissue in human patients that do not contrast 
enhance with traditional MRI imaging. A proportion of non-benign tumors are not 
observable with present MRI imaging techniques. The images in Figure 13 are from a 
patient whose tumor did not contrast enhance with MRI. This lack of enhancement is 

15 usually typical of benign tumors. However, optical imaging was able to identify this 
tumor as a non-benign type (an anoplastic astrocytoma). Figure 13 A shows the gray- 
scale image of the area of interest. Figure 13B shows the difference image prior to dye 
injection. Figure 13C shows the area of interest 1 minute after intravenous dye 
injection, and Figure 13D shows the area of interest 5 minutes after dye injection. Note 

20 that the dye is retained in tins tissue for a significant time. As shown in figures 10, 1 1, 
and 12, tins dynamic trait is a characteristic of a non-benign tumor 

EXAMPLE 10 

This example illustrates imaging of functional regions of peripheral nerves. A 
25 rat is anesthetisedajttf dissected to expose the sciatic nerve. Using diver electrodes we 
electrically stimulate the caudal end of the nerve while acquiring a first sequence of 
difference images. We note the extent of the spread of the intrinsic optical changes in 
the nerve from the point of stimulation by examining the difference imaging containing 
the peak optical change from the control. Next, we make a crush in the nerve at a small 
30 distance anterior to the stimulating electrodes. We acquire a second sequence of 

difference images and compare the corresponding difference image from this sequence 
to the image containing the peak optical change from the first image. We note that the 
intrinsic optical changes diminish abruptly at the point where the nerve was damaged 
Finally, we stimulate the nerve anteriorly to where the crush was made and after 
35 acquiring a third sequence of difference images, we again note where the intrinsic 
changes abruptly end. This method allows us to localize the location and extent of 
damaged or disfunctional peripheral nerve tissue. 
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EXAMPLE 11 

This example illustrates imaging of functional regfans of Cranial Nerve Vm. 
Cranial nerve Vm (Vestibulocochlear nerve) is exposed. Sound tones provide the 
auditory stimulus which eventually cause activation of this nerve. A sequence of 
5 difference images before, during, and after the appropriate auditory stimuli are applied 
show that intrinsic optical changes of the nerve are associated with its functional 
activiarion. Next, a small region of this nerve is damaged by crushing. A second 
sequence of images reveal that auditory stimulation evokes intrinsic optical changes in 
the nerve up to the point of damage. 

10 

EXAMPLE 12 

This example illustrates various methods for enhancing images obtained from 
( tumor tissue or intrinsic signal difference images using multiple wavelength and/or laser 

illumination, and a method for extracting 3-D information using multiple wavelengths. 

15 We expose a region of cortex in an anesthised rat. First, illuminating with white light 
from a tungsten filament lamp, we acquire a sequence of difference images prior to, 
during, and following electrical stimulation of this region of cortex with bipolar 
stimulating electrodes. Next, we acquire second and third difference image sequences, 
following the identical procedure as we did for the first sequence, except that in the 

20 second sequence, the cortex is illuminated with with 690 nm and in the third sequence 
510 nm light. The change in wavelengths is accomplished by placing 690 ± 10 nm 
interference filter or a 5 10 ± 10 nm interference filter between the lightsource and the 
brain. 

We compute the contrast-enhanced image by first ratioing a control 690 nm 
25 image with a control 5 10 nm image. Second, we ratio a 690 nm image during 

stimulation with the corresponding 5 1 0 nm image. We then combine the ratio images 
/ t0 compute the percentage difference image. In this manner, the noise has been 

significantly reduced, hence the signal/noise ratio has been significantly increased. 

Next, we show how to extract depth information from the multiple wavelenght 
SO images that we have acquired. Longer wavelength ligjit penetrates to a greater depth 
through the cortex, and shorter wavelength light to a lesser extent. Hence, the 690 nm 
image as penetrated cortex to x mm, and the 510 nm image to y mm where x <y. 

We subtract the 61 0 nm image from the 5 10 nm image, showing an "optical 
wedge" containing information from a depth of (x -y) mm to x mm within the cortical 
35 tissue. By using a series of other interence filters, we acquire a sequence of images 
containing information from many different depths of the cortex. It is possible to 
acquire 3-dimentional information. 
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Next, exposing tumor tissue in a rat in which we have induced tumor growth, 
we repeat all of the above experiments showing that in a like manner, we can improve 
signal/noise and extract 3-dimentional information in tumor tissue. However, instead of 
stimulating the tissue electrically, we inject the dyes indocyanine green or Evans blue. 

5 Finally, we repeat the above experiments by illuminating the cortex at several 

different wavelengths with a dye-tunable laser (a coherent source) instead of with the 
non-coherent tungsten filament lamp. With the laser (or any coherent source) we have 
the additional advantage in that we can seperate out the components of the signal due to 
changes in reflection or scattering. By illuminating the cortex with the laser directly 

1 0 parallel to the camera (both of which are perpendicular to the brain), we are imaging 
reflected light only. By moving the laser at an angle 6 to the camera, we are measuring 
changes due to scattering alone at this particular angle. 

EXAMPLE 13 

1 5 This example illustrates a C code to implement the inventive algorhithm and 

strategy for automatically translating a pair of images to the best fit with a control 
image with translation in the x-y plane. One can implement the algorithm so that the 
inventive device can automatically translate subsequently acquired images to the control 
image so that motion will be compensated in an on-line fashion in the operating room. 

20 As well, it is clear the this algorithm can be implemented in integer arithmetic so that it 
is computationally efficient. Also, since most of the memory required for this algorithm 
can be dynamically allocated, this algorithm makes efficient use of memory. 

This program automatically translates two images stored in the Imaging 
Technology 151 frame buffers to a best fit according to which translation minimizes the 

25 variance of the subtracted images over the user-selected areas of interests. The user 
specifies an image for frame buffer Bl and then specifies an image for frame buffer 
ALOW to be automatically translated to the Bl image. If the number of areas of 
interest are less than 9 and the search depth is less than 8 then all the data can be read 
from the frame buffer into the host computers RAM This allows for speed and 

30 reduces the 10 to the frame buffer. This program can be easily altered to use integer 
arithmetic only for all calculations. 

This program is compilable under Microsoft's C/C++ Version 7,0 compiler 
linked with Imaging Technology's HEX run-time library. It will run on a PC 486 
compatible host computer controlling a lk x lk frame buffer, an ADI, and an ALU from 

35 Imaging Technology's ITEX 1 51 series hardware. 

^include <stdio.h> 
^include <math.h> 
#include<stdlib.h> 



48 



c 



Wt " 3/25 "' PCT/US93/03S73 

^include <conio.h> 
^include <texl50.h> 
^include <graph.h> 
include <floath> 
5 include <dos.h> 

^define MEM^CHUNK 20 
#define QUIT~ -1 
#defineGO -2 
#define RADIX 10 10 
10 #defme RETURN 13 
#defineE3C 27 
#define CURSORUP 72 
#define CURSOR_DOWN 80 
#define CURSORJUGHT 77 
15 ^define CURSORJLEFT 75 

tfdefine CURSOR_JUMP_UP 141 
^define CURSOR_JUMP_DOWN 145 
#define CURSOR_JUMP RIGHT 116 
#defineCURSOR_JUMPJUEFT 115 
20 struct data_box{ 
intx,y; 

int height, width; 

); 

typedef struct data_box data_box; 
25 int boxjoount = 0; 
int depth = 10; 
void init J>ox_pverlay(void); 

BYTE **ramJ>oxdata(dataJ>ox **map_ptr, int search_depth,mt frame buffer): 
data _box **define_boxraap(void); 
30 datajbox *drawJx>xes(void); 

BYTE **diffJ)ox(dataJ>ox **map_ptr, BYTE **fbl, BYTE **fb2, 
_ int box,jnt x_off; int y_off); 

float sub_rects(dataJ>ox **mapjptr, BYTE **diff_rects, int searchjiepth, 
/ int ptrjplace); 

C 35 BYTE ***diff_map(dataJ>ox **rnap_ptr,int fbl, int £b2, int search depth); 

float *sum_rects(dataJ>ox **mapjjtr > BYTE ••^Hffreets, int sea^wiepth 
,int avjlag); 

int *minJ>oxset(dataJiox **map_ptr > float *£loat_ptr, im search depth)* 
float *var_rects(data_box **mapj>tr, BYTE ***diff rects, int s^rch depth, 
40 intavflag); " ~ 

dataJ>ox **define_boxmap(void) /* returns a pointer to an array of boxes */ 

int i = 0, maxbox = MEM_CHUNK; /* dynamically allocate mem in 20-box */ 
int error Jlag = 0, inchar; /* chunks. */ 

45 dataJ>ox **group_ofbdxes; 
datajbox *box_pointer s 
int watch = GO; 

box_j>ointer » (datajjox *)malloc(maxbox*sizeof(dataj)ox)); 
if{box ^pointer « NULL) 
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printf(^TROUBLE AT 70\ii w ); ' 
group_ofboxes = (data_box **)malloc(niaxbox*sizeof(data_box *)); 
if (group_ofboxes — NULL) 

printf( , '\nTROUBLE AT 74\n"); /* FLAG 1 H! */ 

5 else{ 

while (watch != QUIT){ * 
if(errorJlag = 0) { 

printf( M \n\nTypeESC use drawJ>oxes\n\n M ); 
inchar = getchO; 
10 if (inchar «=0) 

getchO; 
if (ESC — inchar) { 
box_count = 1; 

init_box w overlayO; 
15 while( watch != QUIT) { 

if (i >« maxbox) { 

maxbox += MEM_CHUNK; 

group_ofboxes « (datajjox * *)realioc(group_ofboxes, 
maxbox*$izeo^datajbox *»; 
20 if (group_of J>oxes — NULL) 

printfT'XnTROUBLE AT 91\n u );; /* FLAG 2 WW 

} 

group_of_boxes[H-h] = draw_boxesO; 

printf( H Do you want to draw box number %d ?\n M , 
25 (box_count + 1)); 

inchar = getchO; 

if(inchar=0) 
getchO; 

if(inchar«ESC) 
30 box_count-*-t-; 

else 

watch - QUIT; 

} 

} 

35 } 
} 

} 

ifft < maxbox) { 

group_ofboxes - (datajwx **)reaUocCgroup_of_boxes, 
40 (i)*st2eo^data_box *)); 

if (group jofboxes = NULL) 
printfCWTCOUBLE AT 113\n"); 
} /* FLAG 3 !!!!*/ 

return(group_of boxes); 

45 } 

draw_boxes0 is a simple function that draws boxes on the image for 
the user to view using the overlay capabilities of the ITEX 151 
ADI overlay capability. Its main function is to return a pointer 
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to databox structure containg the information for the location and 
area of the selected area of interest 

int cursjc « 235, curs_y = 220; 
5 data_box *draw boxesfvoid) 

{ " . ' 

int dcursjc,dcurs_y; 
int x_start, y_start, xjength, yjength; 
int text_char,k; 
1 0 char box_number[3]; 

datajbox *boxjpointer; 

box_pointer - (datajxsx *)malloc(si2eof(dataJbox)); 
if(box ^pointer = NULL) 

printf( tt \nTROUBLE AT 13 I\n n ); 
15 line(B2,0,cursjK^,curs^curs_x+4 f curs_y,]); 
line^^.cursjx^curs^-S^urs^curs^y+S,!); 
adiJutmode(DYNAMIC); 
k = 0; 

x_start = y_start = x Jength = y length = 0; 
20 whfle(l){ 

text_char=getchO; 
if (text_char = RETURN) { 
++k; 

dcurs_y=0; 
25 dcurs_x=0; 

if (k = 1) { 

x_start cursjx; 
y_start = curs_y; 

IineCBa^O^curs^x^cursjf^cu^x+^curs^O); 
30 line(B2,0 > curs_x,curs_y-5 > curs_x,curs_y+5,0); 

if(k=-2) 
break; 

} 

35 else if (text_char — 0){ 

text_char - getchQ; 
switch(text_char) { 

case CURSORUP: 
dcurs_y=-l; 
40 dcurs_x==0; 

break; 
case CURSORJDOWN: 
dcurs_y=*l; 
dcurs_jx=0; 
45 break; 

case CURSOR JLEFT: 
dcurs_y=0; 
dcursjx=-l; 
break; 
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case CURSOR_RIGHT: * 
dcurs_y=0, 
dcurs_x=l; 
break; 

5 case CURSOR_JUMP_UP: 

dcurs_y = -7; 
dcurs_x = 0; 
break; 

case CURSOR_JUMPJ30WN: 
10 dcurej^ 7; 

dcurs_x = 0; 
break; 

case CURSOR_JUMP_LEFT: 
dcurs_y = 0; 

15 dcursjx - -7; 

break; 

case CURSOR_JlIMPJRIGHT: 

dcurs_y — 0; 

dcurs_x - 7; 
20 break; 

default: 

dcurs_x - 0; 

dcurs_y = 0; 

break; 

25 } 

> 

else 

text_char= -1; 

if ( (k == 0)&&(text_char != -1) ) { 
30 line(B2 > 0 T cuis_x^ t cui^^curs_x+4 t curs_y > 0); 

line^2 f 0»airs_x,cuTS - ^-5,curs_x,curs fc y+5,0); 
curs_x==inax(min(curs_x+dcurs_x,5 1 1),0); 
curs^^r^ax^n^curs^^dcurs^^?^,©); 
iine(B2 A curs_x-4, curs_y,curs_x+4 ,curs_y> 1 ); 
35 line(B2AcursjK ) curs - y-5,curs x^curs^y+SJ); 

} 

else if (k™ 1) { 

Kne(B2,o;x_start,y_start,x_start + xjength^start,©); 
Ilne(B2,0,x_start,y_start,x_start,y_start + yjength,0); 
40 Kne(B2,0,x_start,y_start + yjength,x_start + 

xjength,y_start + yjength,0); 
Hne(B2A*_start + xjength,yjstar^x_start + 

xjength,y_start + y_length,0); 
curs_x=max(min(curs_x4-dcurs_x,51 1),0); 
45 curs_3FTnax(min(curs_y+dcurs_y,479),0); 
xjength - curs_x - x_start; 
yjength = curs_y - y_start; 

line(B2 J 0 3 x_start i y_start,x - _start + xjength,y_start,l); 
Iine(B2 1 0 1 x_start,y_$tart,x M start ) y_s7art + yjength, 1); 
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line(B2 s 0,x_start s y_start + yjength,x_start + 

xjength,y_start + yjength,l); 
line(B2,0,x_start + xjength,y_start,x_start + 

xjength,y_start + yjength,!); 

5 } 
} 

xjstart = min(x_start,x_start + xjength); 
y_start = mtn(y_start,y_start + yjength); 
xjength = abs(xjength); 
10 yjength ~ abs(yjength); 

_hoa(box_count,box_number,RADIX 10); 
ifl[ (xjength < 10) || (yjength < 10) ) { 
if (xjength > yjength) 

text(B2,0,x_start+xJength/2-7,y__start.l5, 
15 HORIZONTAL, 1 , l,box_number); 

else{ 

if(box_count < 10 ) 

text(B2,0,x_start-ll,y_start4y length/2-2, 
HORIZONTAL, U,boxjtminber); 

20 else 

text(B2,0,x_start-l 8,y_start+y Jength/2-2, 
HORIZONTAL, 1, 1 ,box_number); 

} 

25 else 

text(B2,0 3 x_start+xJength/2-5,y_start+yJength/2-2, 
HORIZONTAL, 1 , 1 ,box_number); 
box j>ointer->x = x_start; /* x coordinate */ 
box_pointer->y - y^start; /* y coordinate */ 
30 box j>ointer->height ~ yjength; /* vertical length */ 
box_pointer->width ~ xjength; /* horiz length */ 
curs_x+=20; /* move cross-hairs to neaiby location */ 
curs_y+=20; 
return box^pointer; 

35 ) 

void init J>ox_overiay(void) 

{ /* Clear B2, set path to Bl f and overlay */ 

fb_setsmask(FRAMEB,OxOOFF); /*B2onBl */ 

fbdfflB^Q); 
40 select_path(Bl); 

adijibanksel(l); 

adiJigroupsel(RED); 

adi_clearlut(250); 

adiJigroupsel(GREEN); 
45 adi_clearlut(0); 

adiJigroupsel(BLUE); 

adi_cleariut(0); 

} 

void init Jtex^stuflftvoid) 
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errjevel(2); 

loadtfg(" w ); 

initsysO; 

} 

BYTE **raniJbaxdata(dataJbox **map_ptr $ int search_depth, int frame J>uffer){ 
int y Jc,xjteit»x jend^Jength^star^y Jength; 
BYTE **imagejrects; 
unsigned int count=0; 
unsigned int totaljength=0; 
select j>ath(firamejbufife); 
for(i=0^<box_count^H-) 

total Jength 4<= (*tnap_ptr[i]). width + 2*search_depth; 
:mage_rects = (BYTE **)malloc(totaIJength*sizeofl[BYTE *)); 
if Omagc^rects NULL) 

printf( M \n409\n M ); 
for<1«0;i<box_count;i-H-){ 

x_start — (*mapjptr[i]).x - searchjdepth; 

xjend = (*map j>tr[i]).x + searchjiepth + (*map_ptrp]).width; 

xjength = (*map_j5tr[i]> width + 2*searcbjdepth; 

y_start = (*map_ptr[i]).y - seardwiepth; 

y Jength = (*mapjptrp3)-height + 2*search_depth; 

for(j =x_starta < x_end; j++){ 

image_rects[count] = (BYTE *)ma!loc(si2eof(BYTE)*y Jength); 
if (image_rects[count] == NULL) 
printfCVnSCREWUP 420\n"); 
fb_rvline(B 1 j ,y_start,yJength,image_rects[count]); 
count++; 

} 

} 

return(image rects); 
} 

BYTE **diffJx>x(dataj3ox **roap_ptr, BYTE **fbjptrl, BYTE **fb_ptr2, 
int box_number,int x_of£; int y_off){ 
unsigned int xjength, y Jength; 
static unsigned int countl as O,count2-0; 
int i j; 

static int old number; 
BYTE **box; 
xjength = (*map < j>tr[box_number]). width; 
yjength = (*mapjptr[box_number]),height; 
if (box_number =0) 

old_number = 0; 
if (old_number != box_number){ 

countl +=■ xjength; 

count2 +« xjength + 2*depth; 

old_number * box_number; 

} 

box - (BYTE **)malloc(sizeof(BYTE *)*xjength); 
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if(box = NULL) 

printf( n \nScrewed at line 341 \n"); 
fiar(H)y<xJength;i+4-){ 

box[i\ - (BYTE *)malloc(sizeof(BYTE)*yJength); 
- if(box[i]=NULL) 

printfC^nScrewed at 362W); 

forO=OJ<yJengthJ«M-) 

box IOD] - (BYTE)( abs(Cint)(fbj>trlD+couiitl][i]) - 

Ont)(fb^tr2ri+x_ofF*-count2]Dty_oflFI) ) ); 

retum(box); 

} 

BYTE ***diff_map(dataJbox **map_ptr,urt fbl, im 9>2, int search depth)! 
BYTE^diflLrect^data.box, **fbl j>tr, **fl>2jrtr; " 
int count= 0; 
intij,k,I; 

unsigned int size, totalize = box_count*(2*search_depth + 1)* 

(2*search depth +1); 
size = 2*search_depth + 1 ; "~ 
dtffjects - (BYTE * * *)malIoc(sizeof(BYTE **)*total size)* 
if (diff_rects ===== NULL) ~ 

print^ B \nScrewed at 379\n tt ); 
fbl_j>tr« ramJboxdata(map_ptr,0,fbl); 
fb2_ptr « ramJboxdata(map - j)tr,search depth,fb2); 
for(i=0;i<box_coimtU++) 

for0=0a<size3++) 

for(k«0*<size*++){ 

diff_rects[count] = diffbox(map _ptr,fl>l j>tr,jfb2_ptr,i j,k); 
count++; 

> 

retum(diff rects); 

} 

float *sum_rects(dataJ>ox **map_ptr, BYTE ***diff rects, int search depth, 

int av_flag){ ~ 
unsigned int y,k,l,m,count=0; 
float size « (2*searchdepth + 1); 

float totdjuxe = (2*search_depth + 1 )*(2*search_depth + l)*box count' I 
float *surh_ptr, ' 

sumjptr - (float *)calloc((unsigned int)totaI size,sizeof(float)y 
if (sumjjtr NULL) 

printfC'^nSCREWUP AT 537\n M )* 
for(i=0;i<box_count;H-h) 
for 0 5 *0;j<size;j++) 
for(k«0;lc<si2e;kHH-){ 
. for(H);l < (*map j>trD]).width; 

for(nr=0;m < (*map_ptr[i]),height^n++) 

sumj)tr[count] diff_rects[count][l][m]; 
if(avJ3ag =* ON) 
sumj)tr[count] - sum jtrfcount]/ 
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((*map_ptr[i]').width * (*map_ptrp]).height); 

oount++; 

} 

return(sum_ptr); 

5 } 

float *var_rects(datajx>x **map_ptr, BYTE ***diflFrects ( int search_depth, 

intavjflag){ 
unsigned int ij,]^l,m,comit=0; 
float size = (2*search_depth + 1); 
10 float totalize ~ (2*search_depth + 1 )*(2*searchjieptb + l)*box_count; 
float *varj>tr,*av_ptr; 
av_ptr = sum_rects(nrap jtr^diffj-eets, search_depth,avjlag) ; 

varjrtr = (float *)caJloc((unsigned int)totel w size,sizeof(float)); 
if(var j>tr — NULL) 
15 printfC^Trouble at 477W); 

for(i=0;i<box_count;ff+) 
for Q=0;j<size;j++-) 
for(k=0;k<size;k++){ 

for(I=0;l < <*map_ptrp3).width; J++) 
20 for(m===0;in < (*mapjptr[i]).hdght;m++) 

varjptr[count] +« ((float)(diflL r ^s[count][I][iii]) 
- av^tr[courrt])*((floaW 

- avj)tr[count]); 

count++; 

25 } 

free(avj>tr); 
return(var_ptr); 

} 

mt *minJ>oxset(data_box **map_ptr, float *float_ptr.int search_depth){ 
SO unsigned int ij t boxJump; 

float metric > min_rnetric = FLT MAX,position=0; 
int shift[3j; 
div_t diy_result; 

box Jump = (2*search_depth + l)*(2*search_depth + 1); 
35 for(i^;i<boxJump;i++){ 
metric — 0; 

foi<j~0y<box_counta++) 

metric += float j>tr[j*boxJump + i]; 
if(metric < min_metric){ 
40 niin_nietric = metric; 

position = i; 

} 

} 

divj-esult « div((int)positiott > (int)(2*search_depth + 1)); 
45 shift[0] « (imposition; 

shift[ 1] = depth - divjesultquot; 
shift[2] * depth - divresultrem; 
retum(shift); 

} 
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int main(void) 
{ 

BYTE ***mockj>ointer; 

charimage[12]; 
5 datajbox **map_pointer; 

int *statsjptr a i; 

float *sum__ptr3; 

int *trans3; 

init hex stuflEQ; 
10 fbJnitOr 

_ciearscreen( _OCLEARSCREEN ); 

_settextposkion( 10, 10); 

select_path(Bl); 

_outtext( tt BASE Image : ••); 
15 scanf( fl %&", image); 

imjread(BlA0,512,4$0,image); 

_settextposkion( 15, 10); 

select jath(ALOW); 

_outtext("Iniage to translate : "); 
20 scarf^^s", image); 

im_read(ALOW t O,0,Sl2,480,irnage); 
js^extposhion(20,20); 
_outtext("Search depth: «); 
scanf( H %d" 5 &depth); 
25 selectj5ath{Bl); 

map jointer = definej>oxmap0; 

sum_ptr3 - var_rects(map jointer,modc_pointer,depth,ON); 
trans3 - min Jboxset(map_pointer,sum jptr3 ,depth); 
printf( B \nVARIANCE : pos = %d, xjfcrans - %d, y trans - %d\n\ 
30 trans3[0],trans3[l] > trans3[2]); 
free(mock_pointer); 
free(suni_ptr3); 
free(trans3); 

return 0; } 
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1. A method for imaging margins and dimensions of solid tumor tissue 
located in an area of interest, comprising: 

5 a. illuminating the area of interest with emr containing the wavelength 

of emr absorbed by a dye; 

b. obtaining a video signal of the area of interest as a sequence of frames 
and processing the sequence of frames into an averaged control image; 

c. administering the dye by bolus injection into vasculature circulating to 
10 the area of interest; 

d. obtaining a series of subsequent frames of the area of interest over 
time and processing the series of subsequent frames into a subsequent averaged image; 

e. comparing each subsequent averaged image with the processed 
averaged control image to obtain a series of difference images; and 

15 £ comparing each difference image for evidence of changed absorption 

within the area of interest which is the outline of solid tumor tissue, wherein tumor 
tissue is characterized by fester absorption of and longer retention of the dye. 

2, The method of claim 1 wherein the dye is selected from the group 

20 consisting of indocyanine green, hematoporphyrin, fluorescein, fluorescamine, NPeg, 
BPD, Evans Blue, and combinations thereof 

3, The method of claim 1 wherein rates of change and magnitude of each 
pixel are compared by: 

25 a. determining a baseline value of each pixel for the wavelength of emr 

absorbed by the dye; 

b. administering the dye; 

c. obtaining a subsequent series of pixel values for the particular 
wavelength of emr, 

30 d. subtractively combining a first averaged image frame from a 

subsequent averaged image to provide a difference image; and 

e. superimposing the difference image onto an analog image. 

4. A method for optically imaging functional and dysfunctional areas of 
35 interest of the cortex in a patient comprising: 

a. illuminating the area of interest with high intensity emr; 

b, obtaining a video signal of the area of interest as a sequence of frames 
and processing the sequence of frames into an averaged control image; 
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c. administering a paradigm to the patient to stiimdate an intrinsic 

signal; 

d. obtaining a series of subsequent frames of the area of interest over 
time and processing the series of subsequent frames into a subsequent averaged image; 

5 e - comparing each subsequent averaged image with the averaged 

control image to obtain a series of difference images; and 

£ comparing each difference image for evidence of an intrinsic signal 
within the area of interest, wherein an intrinsic signal is characterized by a change in 
reflectance properties manifest as a signal in the difference image. 

10 

5. The method of claim 4 wherein the intrinsic signals are detected by an 
apparatus comprising: 

a. a means for obtaining analog video images and a means for 
processing the analog video signals into either an averaged control image or a 

15 subsequent averaged image; 

b. a means for acquiring and analyzing a plurality of subsequent 
averaged images and averaged control image to provide a difference image, wherein the 
difference image is processed to account for movement and noise and amplify the 
changes across a dynamic range of the apparatus; and 

20 a 8 means for displaying the difference image alone or superimposed 

over an analog video image. 

6. The method of claim 5 wherein the means for processing the analog 
video signals into either the averaged control image or the subsequent averaged image 

25 comprises: 

a a means for digitizing the analog video signal and processing the 
digitized image across a full dynamic range; 

b. a means for averaging a series of frames into an averaged image and 
storing data from the averaged image frame in a first frame buffer; 

c. a means for storing each subsequent and processed digitized image 
data in a second frame buffer; 

d. a means for subtractively combining the averaged control image with 
a subsequent averaged image from the first and second frame buffers to provide a 
difference image, said difference image being stored in a third frame buffer; 

e. a means for processing the difference image to stretch the image 
across a full dynamic range; 

f. a means for color-coding the difference image; and 

g. a monitor for viewing a superimposed color-coded difference image. 
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7. An apparatus for imaging tumor tissue comprising: 

a. a means for obtaining analog video images and a means for 
processing the analog video signals into either an averaged control image or a 

5 subsequent image; 

b. a means for acquiring and analysing a plurality of subsequent 
averaged images and an averaged control image to provide a difference image, wherein 
the difference image is processed to account for movement and noise and amplify the 
changes across a dynamic range of the apparatus; and 

10 c, a means for displaying the difference image alone or superimposed 

over an analog video image. 

8. The apparatus of claim 7 wherein the means for processing the analog C 
video signals into either the averaged control image or the subsequent averaged image 

15 comprises: 

a. a means for digitizing the analog video signal and processing the 
digitized image across a full dynamic range; 

b* a means for averaging a series of frames into an averaged control 
image and storing data from the averaged control image in a first frame buffer, 
20 a a means for storing each subsequent and processed digitized image 

data in a second frame buffer; 

d. a means for subtracrively combining the averaged control image with 
a subsequent averaged image from the first and second frame buffers to provide a 
difference image, said difference image being stored in a third frame buffer, 
25 e. a means for processing the difference image to stretch the image 

across a full dynamic range; 

f. a means for color-coding the difference image; and ^ 

g. a means for superimposing the color-coded difference image upon an 
analog video image and a monitor for viewing this image. 
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9. The apparatus of claim 7 or the method of claim 5, wherein the 
comparing steps (e and f) include a means for obtaining a digital image by subtractively 
combining at least two images, wherein the two images are spatially translated relative 
to one another by geometrical transformation. 

10. The apparatus of claim 8 or the method of claim 6, wherein steps (d) and 
(e) are replaced by a means for subtracting a subsequent averaged image from the 
averaged control image to obtain a first difference image and subtracting the averaged 
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control image from the subsequent image to obtain a second difference image, and 
adding the first difference image and the second difference image to create a sum 
difference image showing areas of increased neuronal activity and areas of inhibited 
neuronal activity. 

11. A method for imaging damage to a peripheral or cranial nerve 
comprising: 

(a) illuminating an area of interest with emr, wherein the area of interest 
comprises the peripheral or cranial nerve of interest including the suspected site of 
damage and a region upstream of the suspected site of damage; 

(b) obtaining a video signal of the area of interest as a series of frames 
and processing the series of frames into an averaged control image; 

(c) stimulating the peripheral or cranial nerve at a site upstream of the 
suspected damaged ate; 

(d) obtaining a subsequent scries of frames at the time of stimulation and 
processing the subsequent series of frames into a subsequent averaged image; and 

(e) obtaining a difference image by subtracting the averaged control 
image from the subsequent averaged image to image the active region of the peripheral 
or cranial nerve, whereby nerve blockage is imaged as the point along the nerve where 
the intrinsic signal from the stimulated nerve abruptly ends or is altered in the difference 
image. 

12. A method for imaging a solid tumor tissue located in an area of interest 
wherein the area of interest is located underneath intact skin and/or bone, comprising: 

a. illuminating the area of interest with infrared region of emr, 

b. obtaining a video signal of the area of interest as a series of frames 
and processing the series of frames into an averaged control image; 

c administering a dye by bolus injection into vasculature circulating to 
the area of interest, wherein the dye absorbs emr in a region of the infrared spectrum 
that is capable of penetrating through skin and bone tissue; 

d. obtaining a series of video images of the area of interest over time as 
a subsequent series of frames and processing each subsequent series of frames into a 
subsequent averaged image; 

e. comparing each subsequent frame with the processed averaged 
control frame to obtain a series of difference images; and 

£ comparing each difference image for evidence of changed absorption 
within the area of interest which is the outline of solid tumor tissue, whereby tumor 
tissue is characterized by faster absorption of and longer retention of the dye. 
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13. The method of claim 12 wherein the changes in absorption rates of each 
pixel are compared by; 

a. determining a baseline value of each pixel for the wavelength of emr 
5 absorbed by the dye; 

b. administering the dye; 

c. obtaining a subsequent series of pixel values for the particular 
wavelength of emr, 

d subtract ively combining a first average image frame from a 
JO subsequent image to provide a difference image; and 

e. superimposing the difference image onto an analog image. 

14. A method for imaging functional and dysfunctional areas of interest 
within the CNS, wherein the area of interest is located underneath intact skin or bone, 

15 comprising: 

a* illuminating the area of interest with infrared region of emr; 

b. obtaining a video signal of the area of interest as a series of frames 
and processing the series of frames into an averaged control image; 

c. administering a paradigm to the patient to stimulate an intrinsic 

20 signal; 

d. obtaining a series of video images of the area of interest over time as 
a subsequent series of frames and processing each subsequent series of frames into a 
subsequent averaged image; 

e. comparing each subsequent frame with the processed averaged 
25 control frame to obtain a series of difference images; and 

£ comparing each difference image for evidence of changed absorption 
within the area of interest which is the outline of solid tumor tissue, whereby tumor 
tissue is characterized by faster absorption of and longer retention of the dye. 

30 1 5. The method of claim 14 wherein the changes in absorption rates of each 

pixel are compared by: 

a. detennining a baseline value of each pixel for the wavelength of emr 
absorbed by the dye; 

b. administering the paradigm; 

35 c. obtaining a subsequent series of pixel values for the particular 

wavelength of emr, 

d. subtract ively combining a first average image frame from a 
subsequent image to provide a difference image; and 
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e. superimposing the difference image onto an analog image. 
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